spider.git
3 years agosimply the RBN skimmer scoring system
Dirk Koopman [Sat, 15 Aug 2020 21:45:17 +0000 (22:45 +0100)]
simply the RBN skimmer scoring system

3 years agofix last fix!
Dirk Koopman [Fri, 14 Aug 2020 14:01:07 +0000 (15:01 +0100)]
fix last fix!

3 years agoremove $items warning in RBN
Dirk Koopman [Fri, 14 Aug 2020 13:59:36 +0000 (14:59 +0100)]
remove $items warning in RBN

3 years agofix missing fields in rbn_cache, add timings
Dirk Koopman [Fri, 14 Aug 2020 13:33:19 +0000 (14:33 +0100)]
fix missing fields in rbn_cache, add timings

set/debug rbntimer will show how long a queue takes to process (remember
that this includes sending data out to users).
set/debug rbnskim shows the process of determining the "one true frequency" of
a spot, by running a weighed "auction" of all the spotters's performance
in previous "auctions". A spotter's frequency "score" can be between 5 and -5
(good -> bad) and that is added to each version of the frequency that is
sent by the RBN. The frequency with the highest score is the one posted.
It is the 'QRGScore' in the "progress" spot message.

3 years agonew RBN weighted QRG scoring system.
Dirk Koopman [Thu, 13 Aug 2020 16:52:09 +0000 (17:52 +0100)]
new RBN weighted QRG scoring system.

3 years agoadd uptime cmd, add bells to sh/data
Dirk Koopman [Fri, 7 Aug 2020 15:20:28 +0000 (16:20 +0100)]
add uptime cmd, add bells to sh/data

3 years agoAdd latest RBN chnages and data stats
Dirk Koopman [Thu, 6 Aug 2020 19:13:55 +0000 (20:13 +0100)]
Add latest RBN chnages and data stats

3 years agoAdd CTY-3013 Prefixes
Dirk Koopman [Thu, 6 Aug 2020 18:07:42 +0000 (19:07 +0100)]
Add CTY-3013 Prefixes

3 years agofix mojo complaints in check_cache
Dirk Koopman [Wed, 29 Jul 2020 23:38:16 +0000 (00:38 +0100)]
fix mojo complaints in check_cache

3 years agorbn, ve7cc improvements and fixes
Dirk Koopman [Wed, 29 Jul 2020 22:46:38 +0000 (23:46 +0100)]
rbn, ve7cc improvements and fixes

1. Add show/rbn command that allows one to see who is online and configured
   for RBN. See help sh/rbn for details.
2. Fixed issue with set/ve7cc mode adding unwanted newlines to output.
3. Attempted to make the QRG normalisation more "intelligent". This would be
   sooooooooo much easier if skimmers were calibrated more accurately.
4. Added CTY-3012 prefixes.

3 years agoput sh/rbn into git!
Dirk Koopman [Thu, 23 Jul 2020 15:12:21 +0000 (16:12 +0100)]
put sh/rbn into git!

3 years agoadd sh/rbn command
Dirk Koopman [Thu, 23 Jul 2020 15:09:20 +0000 (16:09 +0100)]
add sh/rbn command

3 years agoFix Filtering, RBN changes
Dirk Koopman [Thu, 16 Jul 2020 22:07:28 +0000 (23:07 +0100)]
Fix Filtering, RBN changes

Filter has changed so that it is now a "simple" transliteration
into perl expressions and relies on perl to do what is required
It is quitely that some more work will be required for error
handling, but the I no longer attempt to do any bracket handling
and leave that up to perl. This means that brackets are fully
working (as opposed to NOT working at all).

Regexes are now robustly translated where necessary. In order to
do this I have ditched Data Dumper because it did not seem to cope
with the regex translations correctly. '\s' would seem have been
translated to '\\s' and not what the regex actually said. So I
have changed the file format to JSON. As this is now the 3rd or
4th JSON change I have made, I have created a "standard" encode and
decode in DXJSON.pm and retrofitted it to all the other places
which had their own purpose built one. DXJSON.pm is just a very
light shim over the standard one.

The RBN changes are:

1. don't short-circuit the zone iteration by a  filter firing.
2. make sure that filtering and not filtering work identically
3. Add constants to the caches records to make it more obvious
   what is going on.

3 years agominor debugging change
Dirk Koopman [Sat, 11 Jul 2020 17:11:48 +0000 (18:11 +0100)]
minor debugging change

3 years agostop crash if node adds itself in pc92
Dirk Koopman [Sat, 11 Jul 2020 14:10:16 +0000 (15:10 +0100)]
stop crash if node adds itself in pc92

3 years agoremove '#' in bands.pl !!!!!!
Dirk Koopman [Sat, 11 Jul 2020 10:23:50 +0000 (11:23 +0100)]
remove '#' in bands.pl !!!!!!

3 years agoa more robust console.pl idle loop fix
Dirk Koopman [Sat, 11 Jul 2020 02:07:44 +0000 (03:07 +0100)]
a more robust console.pl idle loop fix

make all the comparisons 'eq' and check that RESIZE still works

3 years agofix spurious == errors on console.pl input
Dirk Koopman [Sat, 11 Jul 2020 01:56:04 +0000 (02:56 +0100)]
fix spurious == errors on console.pl input

This only happens if it catches a character during the idle loop
which is the only way to catch Curses RESIZE signal. But a stray
"normal" character might get through as well.

3 years agostop dbg init when not in debug mode.
Dirk Koopman [Fri, 10 Jul 2020 22:20:49 +0000 (23:20 +0100)]
stop dbg init when not in debug mode.

This fixes problems reported when running console.pl not as the
sysop user.

3 years agokill some 5.28 warnings
Dirk Koopman [Thu, 9 Jul 2020 22:24:28 +0000 (23:24 +0100)]
kill some 5.28 warnings

3 years agoadd --all to the git pill in UPGRADE.mojo
Dirk Koopman [Thu, 9 Jul 2020 21:29:22 +0000 (22:29 +0100)]
add --all to the git pill in UPGRADE.mojo

3 years agoUpdate the docs for the latest version
Dirk Koopman [Thu, 9 Jul 2020 21:19:41 +0000 (22:19 +0100)]
Update the docs for the latest version

* major change: convert the users file to Json.
* NEW: add an RBN client.

3 years agoMerge branch 'users.v3j' into mojo
Dirk Koopman [Thu, 9 Jul 2020 20:10:07 +0000 (21:10 +0100)]
Merge branch 'users.v3j' into mojo

3 years agoadd warning about filters and set/skimmer users.v3j
Dirk Koopman [Wed, 8 Jul 2020 23:55:43 +0000 (00:55 +0100)]
add warning about filters and set/skimmer

3 years agofix help for priv 9 set/skimmer
Dirk Koopman [Wed, 8 Jul 2020 23:42:47 +0000 (00:42 +0100)]
fix help for priv 9 set/skimmer

3 years agorm rbn from Aliases, change help
Dirk Koopman [Wed, 8 Jul 2020 22:58:42 +0000 (23:58 +0100)]
rm rbn from Aliases, change help

Add a callsign to invalid characters in command handling

3 years agoThe last revision before merge back to mojo?
Dirk Koopman [Wed, 8 Jul 2020 22:01:00 +0000 (23:01 +0100)]
The last revision before merge back to mojo?

See Changes file for details

3 years agofix (bandaid for now) sh/cl local counts
Dirk Koopman [Tue, 7 Jul 2020 17:56:52 +0000 (18:56 +0100)]
fix (bandaid for now) sh/cl local counts

3 years agofix (bandaid for now) sh/cl local counts
Dirk Koopman [Tue, 7 Jul 2020 17:56:52 +0000 (18:56 +0100)]
fix (bandaid for now) sh/cl local counts

3 years agofix show/node
Dirk Koopman [Tue, 7 Jul 2020 17:25:31 +0000 (18:25 +0100)]
fix show/node

3 years agobackport console.pl from users.v3j
Dirk Koopman [Mon, 6 Jul 2020 16:20:32 +0000 (17:20 +0100)]
backport console.pl from users.v3j

3 years agomore RBN.mojo changes
Dirk Koopman [Mon, 6 Jul 2020 16:11:13 +0000 (17:11 +0100)]
more RBN.mojo changes

3 years agomore RBN.mojo changes
Dirk Koopman [Mon, 6 Jul 2020 15:23:28 +0000 (16:23 +0100)]
more RBN.mojo changes

3 years agoupdate RBN.mojo
Dirk Koopman [Mon, 6 Jul 2020 15:11:54 +0000 (16:11 +0100)]
update RBN.mojo

3 years agoadd current node to console.pl
Dirk Koopman [Mon, 6 Jul 2020 14:32:29 +0000 (15:32 +0100)]
add current node to console.pl

3 years agoadd RBN.mojo
Dirk Koopman [Mon, 6 Jul 2020 01:02:45 +0000 (02:02 +0100)]
add RBN.mojo

3 years agoattempt to prevent echoing on ExtMsg login
Dirk Koopman [Sun, 5 Jul 2020 17:23:32 +0000 (18:23 +0100)]
attempt to prevent echoing on ExtMsg login

This is to try to prevent any PROXY message being echoed

3 years agoAdd HAProxy type 1 "real ip" address changes
Dirk Koopman [Sun, 5 Jul 2020 15:21:43 +0000 (16:21 +0100)]
Add HAProxy type 1 "real ip" address changes

3 years agofix show/dxcc
Dirk Koopman [Sun, 5 Jul 2020 00:52:19 +0000 (01:52 +0100)]
fix show/dxcc

3 years agofix show/dxcc
Dirk Koopman [Sun, 5 Jul 2020 00:52:19 +0000 (01:52 +0100)]
fix show/dxcc

3 years agofix console.pl
Dirk Koopman [Sun, 5 Jul 2020 00:27:02 +0000 (01:27 +0100)]
fix console.pl

Including removing the 80-1 limit on local connections, fixing the
scrolling and generally tarting console.pl up.
Change dx spot RBN format to "traditional" '-#' format.

3 years agofix longstanding issues with console EOL handling
Dirk Koopman [Sat, 4 Jul 2020 01:03:29 +0000 (02:03 +0100)]
fix longstanding issues with console EOL handling

Historically there has been an issue with line endings in console.pl
which meant that the spot line (for instance) had to be 1 character
less than the window width (e.g. 79 chars in an 80 char window).

This has now FINALLY been fixed.

In the process I have fixed the issues of extra things like
cq/itu/us state/dxgrid strings overwriting or coming out in a
stupid priority. So the priority is now from nearest/smallest
area -> largest wiz: QRA, US State, ITU or CQ zone. So you
can have them all on, but ITU zone will always trump CQ zone.

3 years agoimprove dbg tagging and dbgdump, Add call to conns
Dirk Koopman [Fri, 3 Jul 2020 23:20:41 +0000 (00:20 +0100)]
improve dbg tagging and dbgdump, Add call to conns

It would appear that the $conn did not always have a callsign set
on incoming connections. It does now.

3 years agoadd RBN stats + other fixes
Dirk Koopman [Tue, 30 Jun 2020 12:33:33 +0000 (13:33 +0100)]
add RBN stats + other fixes

Add some more comprehensive RBN stats (per minute, 10 minutes, hour).
Make the RBN spot cache global and make the inrush preventor code
work only on the first connection to skimmer provider.
Fix the $_isdbg clearing so that non-conditional logging messages
come out with a (*) in watchdbg and not the previous isdbg() sort.

3 years agoMerge branch 'mojo' into users.v3j
Dirk Koopman [Sun, 28 Jun 2020 14:14:44 +0000 (15:14 +0100)]
Merge branch 'mojo' into users.v3j

Also convert QSL.pm and create_qsl.pl to JSON format.

3 years agofix ipv4 addresses masquerading as ipv6
Dirk Koopman [Sun, 28 Jun 2020 12:01:54 +0000 (13:01 +0100)]
fix ipv4 addresses masquerading as ipv6

3 years agovarious RBN fixes, fix links spacing
Dirk Koopman [Thu, 25 Jun 2020 14:00:34 +0000 (15:00 +0100)]
various RBN fixes, fix links spacing

Add incoming RBN filtering from the RBN servers.
Use spot filtering if there is no RBN specific filtering.
Fix formatting issues in the 'links' command for isolated and skimmer nodes.

3 years agouse spot filters for rbn in no rbn filters exist
Dirk Koopman [Tue, 23 Jun 2020 22:32:45 +0000 (23:32 +0100)]
use spot filters for rbn in no rbn filters exist

3 years agomake sure wantrbn is picked up again on restart
Dirk Koopman [Mon, 22 Jun 2020 18:21:33 +0000 (19:21 +0100)]
make sure wantrbn is picked up again on restart

remove words about convert-v3-to-v4.pl from UPDATE.mojo

3 years agofix new RBN spot typos
Dirk Koopman [Sun, 21 Jun 2020 14:05:55 +0000 (15:05 +0100)]
fix new RBN spot typos

3 years agoRBN do filtering one per stored spot not per user
Dirk Koopman [Sat, 20 Jun 2020 22:27:55 +0000 (23:27 +0100)]
RBN do filtering one per stored spot not per user

3 years agoadd RBN filtering
Dirk Koopman [Sat, 20 Jun 2020 22:04:10 +0000 (23:04 +0100)]
add RBN filtering

3 years agoSet $Spot::readback back to 1
Dirk Koopman [Wed, 17 Jun 2020 11:59:03 +0000 (12:59 +0100)]
Set $Spot::readback back to 1

This is to solve the problem of losing found spots when traversing
multiple files.

TO BE FIXED!! So we can read forward again.

3 years agofix respotted spots
Dirk Koopman [Thu, 11 Jun 2020 14:26:18 +0000 (15:26 +0100)]
fix respotted spots

3 years agorbn collect qra info ftom FTx spots
Dirk Koopman [Thu, 11 Jun 2020 14:12:25 +0000 (15:12 +0100)]
rbn collect qra info ftom FTx spots

3 years agoincrease rbn connection startup delay
Dirk Koopman [Thu, 11 Jun 2020 13:08:50 +0000 (14:08 +0100)]
increase rbn connection startup delay

3 years agomake rbn queues local, improve qrg jitter detect
Dirk Koopman [Thu, 11 Jun 2020 12:45:19 +0000 (13:45 +0100)]
make rbn queues local, improve qrg jitter detect

3 years agoRBN "fix" qrg variation, add extra flags
Dirk Koopman [Thu, 11 Jun 2020 10:34:01 +0000 (11:34 +0100)]
RBN "fix" qrg variation, add extra flags

* = multiple QRGs
+ = respot

3 years agoadd missing set/rbn.pl
Dirk Koopman [Thu, 11 Jun 2020 00:20:55 +0000 (01:20 +0100)]
add missing set/rbn.pl

3 years agoLatest version of RBN code
Dirk Koopman [Thu, 11 Jun 2020 00:18:03 +0000 (01:18 +0100)]
Latest version of RBN code

3 years agolengthen trailing numeric part of prefix
Dirk Koopman [Mon, 8 Jun 2020 16:38:43 +0000 (17:38 +0100)]
lengthen trailing numeric part of prefix

to allow things like GB5000GCM etc

3 years agoadd better errors in DXUser::Exportusers
Dirk Koopman [Mon, 8 Jun 2020 16:40:17 +0000 (17:40 +0100)]
add better errors in DXUser::Exportusers

3 years agolengthen trailing numeric part of prefix
Dirk Koopman [Mon, 8 Jun 2020 16:38:43 +0000 (17:38 +0100)]
lengthen trailing numeric part of prefix

to allow things like GB5000GCM etc

3 years agoMerge branch 'mojo' into users.v3j
Dirk Koopman [Mon, 8 Jun 2020 16:19:11 +0000 (17:19 +0100)]
Merge branch 'mojo' into users.v3j

just to keep stuff ticking along

3 years agofix erronoeus input filter cmd causing crashes
Dirk Koopman [Mon, 8 Jun 2020 15:19:09 +0000 (16:19 +0100)]
fix erronoeus input filter cmd causing crashes

3 years agofix sh/mydx, add back qra sq for sh/dxgrid
Dirk Koopman [Mon, 8 Jun 2020 13:49:27 +0000 (14:49 +0100)]
fix sh/mydx, add back qra sq for sh/dxgrid

3 years agofix beacons on RBN
Dirk Koopman [Sun, 7 Jun 2020 01:04:07 +0000 (02:04 +0100)]
fix beacons on RBN

3 years agoput back startup_delay to RBN
Dirk Koopman [Sun, 7 Jun 2020 00:53:00 +0000 (01:53 +0100)]
put back startup_delay to RBN

3 years agofix create_prefix.pl
Dirk Koopman [Sun, 7 Jun 2020 00:49:59 +0000 (01:49 +0100)]
fix create_prefix.pl

3 years agolatest version of RBN. Add qra to prefixes.
Dirk Koopman [Sun, 7 Jun 2020 00:18:27 +0000 (01:18 +0100)]
latest version of RBN. Add qra to prefixes.

Also fix small issue with reading a line from an sql file and
include the ip address.

3 years agoadd <tr>,<es> and <ms> shortcuts to sh/dx
Dirk Koopman [Wed, 3 Jun 2020 23:14:47 +0000 (00:14 +0100)]
add <tr>,<es> and <ms> shortcuts to sh/dx

3 years agoget rid of sh/dx debugging warnings
Dirk Koopman [Wed, 3 Jun 2020 22:56:24 +0000 (23:56 +0100)]
get rid of sh/dx debugging warnings

3 years agofix log and sh/dx ordering issues.
Dirk Koopman [Wed, 3 Jun 2020 22:38:27 +0000 (23:38 +0100)]
fix log and sh/dx ordering issues.

3 years agoadd chenges to sh/dx
Dirk Koopman [Wed, 3 Jun 2020 21:08:30 +0000 (22:08 +0100)]
add chenges to sh/dx

3 years agofix regex parse in sh/dx, add DXBearing::distance
Dirk Koopman [Wed, 3 Jun 2020 19:42:27 +0000 (20:42 +0100)]
fix regex parse in sh/dx, add DXBearing::distance

3 years agofix the small whoopsie in sh/dx
Dirk Koopman [Tue, 2 Jun 2020 20:42:02 +0000 (21:42 +0100)]
fix the small whoopsie in sh/dx

3 years agofinal(!) issued version of sh/dx
Dirk Koopman [Mon, 1 Jun 2020 14:28:45 +0000 (15:28 +0100)]
final(!) issued version of sh/dx

3 years agofix !( spurious && before next item in Filter
Dirk Koopman [Sun, 31 May 2020 16:42:42 +0000 (17:42 +0100)]
fix !( spurious && before next item in  Filter

and some more RBN WIP

3 years agoinitial removal of Storable
Dirk Koopman [Sun, 31 May 2020 13:59:02 +0000 (14:59 +0100)]
initial removal of Storable

3 years agoand the necessary mod for testing / in calls
Dirk Koopman [Sun, 31 May 2020 13:54:55 +0000 (14:54 +0100)]
and the necessary mod for testing / in calls

Add parameter to allow testing of PA0/G1TLH style callsigns

3 years agoadd missing Changes file
Dirk Koopman [Sun, 31 May 2020 13:54:01 +0000 (14:54 +0100)]
add missing Changes file

3 years agofix sh/dx by call and missing files
Dirk Koopman [Sun, 31 May 2020 13:49:56 +0000 (14:49 +0100)]
fix sh/dx by call and missing files

Add a basic implimentation of allowing callsigns like PA0/G1TLH
to login as normal users that are completely distinct from their
(apparent) parent callsign. PA0/G1TLH has no connection with
G1TLH despite appearances.

3 years agoimprove links cmd. fix UPGRADE.mojo
Dirk Koopman [Sun, 31 May 2020 10:43:39 +0000 (11:43 +0100)]
improve links cmd. fix UPGRADE.mojo

3 years agofix info xxxx
Dirk Koopman [Sat, 30 May 2020 17:04:36 +0000 (18:04 +0100)]
fix info xxxx

3 years agochange sh/dx backend to the Filter engine
Dirk Koopman [Sat, 30 May 2020 16:46:24 +0000 (17:46 +0100)]
change sh/dx backend to the Filter engine

3 years agoadd an RBN line to progress
Dirk Koopman [Thu, 28 May 2020 20:35:34 +0000 (21:35 +0100)]
add an RBN line to progress

3 years agoadd new colour for RBN in console
Dirk Koopman [Wed, 27 May 2020 23:43:43 +0000 (00:43 +0100)]
add new colour for RBN in console

3 years agoRBN now with basic spots
Dirk Koopman [Wed, 27 May 2020 23:35:42 +0000 (00:35 +0100)]
RBN now with basic spots

3 years agonew dbgtrace command
Dirk Koopman [Wed, 27 May 2020 20:45:00 +0000 (21:45 +0100)]
new dbgtrace command

Avoid the use of Carp long/short_message and replace it with our own.

3 years agoWIP RBN
Dirk Koopman [Tue, 26 May 2020 08:18:46 +0000 (09:18 +0100)]
WIP RBN

3 years agofix the issue in DXProt::add_thingy
Dirk Koopman [Tue, 26 May 2020 18:53:13 +0000 (19:53 +0100)]
fix the issue in DXProt::add_thingy

For some reason I wasn't checking whether PC92 routing message was valid
enough. I shall probably have to harden it even more still.

Also fixed DXDebug::printdbgring(nn) so that actually honours nn!

3 years agoadd connect time/uptime for links cmd
Dirk Koopman [Tue, 26 May 2020 08:18:01 +0000 (09:18 +0100)]
add connect time/uptime for links cmd

3 years agoAllow stuff on the console with no disk logging
Dirk Koopman [Sat, 23 May 2020 23:03:21 +0000 (00:03 +0100)]
Allow stuff on the console with no disk logging

3 years agolimit the no of spots stored for a search
Dirk Koopman [Sat, 23 May 2020 17:37:49 +0000 (18:37 +0100)]
limit the no of spots stored for a search

This is to cope with people that suck skimmer spots and end up
with 100Mb files to read. But it does rely on the hint getting
most of the candidates with not more than 2 false positives.

3 years agofix DXSubcommand thaw protection
Dirk Koopman [Sat, 23 May 2020 15:30:36 +0000 (16:30 +0100)]
fix DXSubcommand thaw protection

3 years agotry to fix DXSubprocess errors
Dirk Koopman [Sat, 23 May 2020 13:51:09 +0000 (14:51 +0100)]
try to fix DXSubprocess errors

That are happening at the EA3CV-2.
Also make sure that an incoming connection's IP address is recorded.

3 years agoFix the DXC locking out DXC-5 issue
Dirk Koopman [Thu, 21 May 2020 16:25:51 +0000 (17:25 +0100)]
Fix the DXC locking out DXC-5 issue

3 years agoAdd CTY-3010 prefixes
Dirk Koopman [Wed, 20 May 2020 17:49:12 +0000 (18:49 +0100)]
Add CTY-3010 prefixes

3 years agoadd gitbranch to pc18
Dirk Koopman [Wed, 20 May 2020 15:27:11 +0000 (16:27 +0100)]
add gitbranch to pc18

3 years agoshow git branch in show/version
Dirk Koopman [Wed, 20 May 2020 15:10:02 +0000 (16:10 +0100)]
show git branch in show/version