X-Git-Url: http://dxcluster.net/gitweb/gitweb.cgi?a=blobdiff_plain;f=Changes;h=4161d2ac4a6d34e7b39e04abced9ec2ffcb4900f;hb=refs%2Fheads%2Fmojo;hp=f0dd66b2c29dac4c69a90ac923bfdf6d68fe1481;hpb=56fd7417545aa9de3101434bc88156b5eebb37ab;p=spider.git diff --git a/Changes b/Changes index f0dd66b2..4161d2ac 100644 --- a/Changes +++ b/Changes @@ -1,3 +1,391 @@ +20Apr24====================================================================== +1. Try to "fix" Spot deduping (again). This is probably Work In Progress... +2. A new(er) band.pl gratefully received from Andy G4PIQ. +13Apr24====================================================================== +1. Fix small issue with perl 5.20 (and possibly earlier) compilation. +27Mar24====================================================================== +1. Fix dupe PC[16]1 (spot) coming in "too fast" for dupe filter to work. + + Probably. +25Mar24====================================================================== +1. Make talk work. Note: this now works as the mojo branch originally + expected - which means that it may (and likely will) flood route talks + where the callsign that you want to talk to does not have a route to + both the callsign and any nodes that it may be connected to. +22Mar24====================================================================== +1. Change the processing of trailing callsigns slightly so that /1 /2 etc are + retained, but the /P /M /A /MM etc are removed. +20Mar24====================================================================== +1. Has it really been so long since the last update? +2. Since forever, mainly because filtering was done very early on in + DXSpider's development, clearing a filter needed a disconnect/reconnect + cycle from the user/node to activate. Now clearing or changing a filter is + immediately effective in the current session. + + Also things like: 'rej/spot user_default on vhf' will immediately apply + to all users that do not already have some other filter in place. Users + can override this by setting a filter. + + The Filter system will now show the actual token that a faulty command + dislikes. +3. Disable the "is there a route to callsign X3XXX" test in talk. This is a + legacy test that should not apply at the user level. The resulting PC93 is + flood routed unless there is a genuine direct route to that callsign. +25Mar23======================================================================= +1. Changed the default of X and Y (in item 2 below) to 27 and 20 respectively. +24Mar23======================================================================= +1. Restore perl regex version of is_ipaddr. This *should* allow windows + updates to work again. +2. Added cmd entry rate limiting. If a user sends X commmands in Y secs then + they are disconnected without notice. The defaults are X + ($DXCommandmode::maxcmdcount) = 16 and Y ($DXCommandmode::cmdinterval) = 9. + + These can be changed by the startup script. + + These default values are set generously deliberately to allow certain user + programs to get with the program and reduce the number of cmds that they + issue on connection down to something reasonable. For instance, I cannot + see why things like name, qth, lat/long/QRA (amongst several other sticky + user attributes that only need to be entered once) are sent on every login. + + These default values WILL reduce over time. +23Mar23======================================================================= +1. Fix DXCIDR brokenness introduced recently, speeded up DXCIDR::sort, allow + sh/badip 2a05:dfc7:402a::1 45.134.225.96 159.69.38.160 to test individual + IP address as well as show the default (of every badip). +2. Speed up and be more strict with input for is_ipaddr(). +22Mar23======================================================================= +1. Fix warning in DXUtil::difft. +21Mar23======================================================================= +1. Fix DXUtil is_ipaddr function which was accidentally included in previous + commit. +2. Do not extract IP addresses to create routing info as default. +17Mar23======================================================================= +1. Try to nail down incoming PC92s that attempt to change the type of $myalias + or $mycall. +2. Do some housekeeping with the startup and finish debugging to make the + order of messaging more logical. Also disable some messaging caused by + other errors/circumstances causing various object handles disappearing. +3. Make sure that $mycall and $myalias are upper case (and moan about it if + they aren't. +10Mar23======================================================================= +1. I am aware that there are windows nodes out there on mojo (brave, impetuous + that you are) and wget is not an easy option to download badip files. So, + through the power of mojo I give you the new 'download' command. As it + stands at the moment, it will download a file from a webserver and place + it in /spider/local_data. If it downloads successfully, it will overwrite + any file with that name in /spider/local_data. Needless to say this is a + sysop only command. + +24 * * * * run_cmd('download http://www.dxspider.net/download/badip.torexit') +24 * * * * run_cmd('download http://www.dxspider.net/download/badip.torrelay') +24 * * * * run_cmd('download http://www.dxspider.net/download/badip.global') +25 * * * * run_cmd('load/badip') + + would be an obvious use. But it can also be used to download spot, USDB + and other data as well. If you do use it then you will still need to + do a spawn_cmd in crontab for the conversion programs, but that too is + going to change. +07Mar23======================================================================= +1. Fix odd (and old) regression dx command (line ending 59+++) '++' being the + culprit. +2. Allow multiple commands on one line when separated with the 2 character + string '\n' (sigh). +3. Move a few init things about on startup so that more parameters are + scriptable. +4. Put some debug comments before loading various caches (that might take a + longgggg time) and then tell people how long each cache load took. This + should reduce the worry that something is "wrong" when there is a delay + (such as searching for announces). +06Mar23======================================================================= +1. Fix filter error reporting, including incrementing concurrent error count + if there are actually any detected parse errors. +2. Fix warnings on difft. +3. Add the origin of any incoming SP if it is emailed to the recipient. +4. Oops I accidently restored one path of the node being disconnected that I + fixed below (because of changes to filtering). +04Mar23======================================================================= +1. Fixed regression caused by too many command errors in (startup) script + files. This is caused by much stricter checking of commands entered both + by users AND scripts. If there more concurrent errors ie. greater than + $DXChannel::maxerrors (default 10) then that user will be disconnected. + + Unfortunately this could include the $mycall (node) channel if there were + too many command users in the startup script. The example I looked at had + a load of HTML pasted into it. + + For various reasons the only manifestation of this problem was that pings + no longer worked. + + This has been fixed. I am grateful to Fulvio HB9DHG for letting me break + his node for an hour or so until I worked out what was going on. +03Mar23======================================================================= +1. Fix DXDebug::DXDebug not found errors in rarely used functions. This is + also an interaction with differing versions of Mojolicious. Clearly this + has never happened to me (sigh). +2. Allow non-privileged users to use ping (no arguments) and get 'PONG nn' + where nn is an incrementing counter or if there is an argument it will + return 'PONG ' - all in UPPER CASE. See help ping for more + details. +02Mar23======================================================================= +1. Fix sh/dx/30 problem. I.e. allow old style version of sh/dx 30. +01Mar23======================================================================= +1. Revert PC92 A D and PC92 C record flags back to defaults. +2. Deal with users that send gibberish. +3. Fix show/425 (default) URL. +4. Improve Spot deduping. +28Feb23======================================================================= +1. Default ip addresses ON in PC92 C records, disable PC92 A D record output. +2. Add CTY-3308 prefixes. +3. Fix gen_usdb_data.pl (add missing use DXUtil), update current location of + official FCC databases. +4. Remove the duplicate IP addresses across exits and relays from badip lists + during generation by create_master_badip_files.pl. +14Feb23======================================================================= +1. Attempt to be more M$ Windows compatible. This basically is to do with the + unfortunate fact that most of the Windows perl cannot do, or simulate + spawning (running another process in parallel) and/or Mojolicious cannot + handle coping with one or more versions of these simulations. + + I don't have a windows perl available to me at moment so I can only + simulate running under windows by setting a variable. + + You need to know that running any version of DXSpider on Windows will do + all long running commands in line. Much as the 'master' branch does. So + running large nodes on Windows boxes with versions of perl that do not + support running spawned processes continues to be contra-indicated. Much + as it has ALWAYS been. +04Feb23======================================================================= +1. Fixed sh/log so that callsigns beginning with a digit (or several) are + printed rather than being ignored. + This affects: show/ann, chat, log, rcmd and wx +03Feb23======================================================================= +1. Fix is_ipaddr to accept trailing '::' on IPV6 addresses. +2. Fix and extend the TEST program 'showdx' so that it now works on the mojo + branch. Type 'showxd -?' for more information or read the source. This is + NOT a user program, but is will accept most sh/dx expressions. +3. Fix create_master_badip_files.pl so that it does not emit IP addresses + that the system is_ipaddr() function fails. +01Feb23======================================================================= +1. Harden DXCIDR (badip stuff) against format errors in downloaded badip files + downloaded using wget from the crontab. If these problems persist PLEASE + TELL me and send me examples of the errors that end up in the badip files. +30Jan23======================================================================= +1. Add ip addresses to outgoing PC93 messages +2. Get rid of (some of?) the uninitialised warnings +3. Add the ability to set the (time) locale to UTC or LOCALTIME by adding + the string LOCALE = UTC or LOCALE = LOCAL(TIME)? to your local crontab + + NOTE: this will affect ALL timings in both the system as well as your + local crontabs. Changing the LOCALE will make no difference to the system + crontab other than using localtime rather than UTC. Which is fine by me. + + You can see what crontab is doing and which LOCALE is in use by doing a + 'set/debug cron' in a console and running a 'watchdbg cron' in another + shell. +25Jan23======================================================================= +1. Add a switch to enable the routing of generated PC92 A/D records and also + prevent the rebroadcast of incoming ones. + + set/var $DXProt::pc92_ad_enable 0 + + to disable PC92 A/D records. +2. Add a switch to enable the addition of IP addresses to PC92 C records. + + set/var $DXProt::pc92c_ipaddr_enable 1 + + to enable the addition of IP addresses to outgoing PC92 C records. + + ## These two features are experimental, but will become the default if + ## some other changes and these are successful +24Jan23======================================================================= +1. Optimise the PC11->PC61 promotion code, improve readability of the + debugging when 'set/debug pc11' is active. +2. Remove spurious '$' character at the end of Listeners.pm (thanks Kin). +3, Fix show/ik3qar command +20Jan23======================================================================= +1. Add the variable @main::localhost_names to allow other IP addresses to + be treated in the same way as localhost in item 1 on 19Jan23 below. NOTE + you must include ALL the normal localhost names + any other interface + names that you might want to include: + + set/var @main::localhost_names qw(127.0.0.1 ::1 192.168.1.30) + + using the qw() construction is easier than: + + set/var @main::localhost_names ('127.0.0.1', '::1', '192.168.1.30') + + but either will work. You can define as many IP addresses as you like and + they can be IPV4 or 6. + + You do NOT need to fiddle with this unless you specifically have more + than just the normal definitions of localhost. So for 'normal' nodes with + one external interface, you DO NOT NEED TO DO ANY OF THIS. +2. Added CTY-3304 prefix data +3. Fix Bad IP address system so that one can carry on as normal without + Net::CIDR::Lite (or libnet-cidr-lite-perl on debian based systems) +19Jan23======================================================================= +1. Introduce aliasing for localhost in DX Spots and outgoing PC92 A records + on login. There are two variables which can be set with the alias to use: + $main::localhost_alias_ipv4 + $main::localhost_alias_ipv6 + These can be set in the /spider/scripts/startup, but this is only + necessary if the node has more than one interface, or virtual hosts. If + there is ONLY ONE ipv4 and/or ipv6 IP address on the node machine then + these variables will be automatically populated on first use. But the SAFE + thing to do is to set them in the startup file. + + THIS FEATURE IS EXPERIMENTAL... +18Jan23======================================================================= +1. Make sure than *every* channel has an IP address. Thank you (I think) Kin + for pointing out that PC92 A records were not going out with IP addresses. + I'm guessing that other things (like spots) had a similar problem. +15Jan23======================================================================= +1. Fix strange errors for carp on missing route_*_cache files on startup. +14Jan23======================================================================= +1. Fixed route PC11 promotions so that a new PC61 is actually generated and + also sent instead of the original PC11 (to PC61 capable nodes). +13Jan23======================================================================= +1. Periodically store Routing tables and, if they are young enough (def: 3hrs) + autotically restore them on restart of the node. This will short circuit + the need to rebuild the routing tables from scratch on every restart - + which is normally for something like software update. +2. Fix pc11 debugging stats with the correct figures. Sigh... Also move some + of the totals to a different place. +3. Add show/spotstats command which gives the current spot statistics shown + during pc11 debugging (which means you don't need to set/deb pc11 unless + you really want that extra noise). +12Jan23======================================================================= +1. Regularise 'set/debug pc11' output to track all the routes through PC11 and + PC61 processing and statistics. +11Jan23======================================================================= +1. Improve (?) the PC11 -> PC61 upgrading process that delays incoming PC11s + for a very short time in the hope that a PC61 will come in to be used + instead. It will also upgrade a PC11 if we have an uptodate IP address + that has come in from the routing system PC92s. do a 'set/debug pc11' to + see it in action. +2. I have chosen a definitive list of TOR exits and relays which can be + downloaded from http://www.dxspider.net/download/badip.torexit, + http://www.dxspider.net/download/badip.torrelay and finally, for those IP + addresses that are deemed to be 'bad': + http://www.dxspider.net/download/badip.global. I have added the following + lines to my /spider/local_cmd/crontab: + +24 * * * * spawn('cd /spider/local_data; wget -qN http://www.dxspider.net/download/badip.torexit') +24 * * * * spawn('cd /spider/local_data; wget -qN http://www.dxspider.net/download/badip.torrelay') +24 * * * * spawn('cd /spider/local_data; wget -qN http://www.dxspider.net/download/badip.global') +25 * * * * run_cmd('load/badip') + + The tor files are downloaded from https://lists.fissionrelays.net/tor/ at + 15 minutes past every hour, please would you use some other minute than + 23 or 24 to get your own local copies. + + A 'set/debug badip' will show you what is being blocked. +3. Fix set/badip so that it appends new IP addresses correctly. +10Jan23======================================================================= +1. Add baddx on incoming callsign in RBN. +2. Search for all /spider/local_data/badip.* files to allow more control on + which IP addresses are detected. e.g. badip.torexit, badip.torrelay as well + as baddx.local. The suffixes, apart from .local (created by set/badip) are + completely arbitrary. You can use whichever suffix name you like. This is + a more useful arrangement for the ever increasing sources of "bad ip + addresses" that we need to deter. + + NOTE: all badip. are read only EXCEPT badip.local (which can be + altered in real time by the sysop using set/badip ...). + If one uses periodic crontab jobs to update any other badip. + files from web resources then don't forget to 'load/badip' afterwards. +3. Add a /spider/data/baddx.issue file which can be copied to (or used as a + basis to create) /spider/local_data/baddx +09Jan23======================================================================= +1. Finish implemention of DXCIDR ip address filtering. This works on both + logins (treated the same as locked out - i.e. just disconnected) and also + with PC61s where these sentences are just dropped. Also attempt to prevent + any *following* PC11s with the same data getting through. + + YOU WILL NEED either 'cpanm Net::CIDR::Lite' or debian/ubuntu based distros + 'apt install libnet-cidr-lite-perl'. RedHat based systems will have similar + packages available. + +2. Recognise PC18s coming from CC Clusters more nicely. +04Jan23======================================================================= +1. Fillout DXCIDR, attach checks in PC61 and logins. Login that fail will + simply disconnect, like locked out callsigns +2. Fix qrz.com URL in stock Internet.pm. +3. Fix DXHash issues (baddx, badnode, badspotter etc) +30Dec22======================================================================= +1. Add more BadWords (regex) checks. +01Dec22======================================================================= +1. Re-add some debugging to see which incoming PC protcol sentences are + being dumped because of any bad content (words or calls) if debugging + option 'nologchan' is set. +2. All debugging for badwords has the word 'badword' in it. So that one can + do a 'grepdbg -1 badword' and one should see who said it and all of what + they said. Watchdbg will also work with this. +3. My latest recommended additions to the badword list includes: + RU STOP SPOT DISGRACE CRM CRIM CRUD SUBHUMAN FASCIST DEGENERATE CATTLE CNT +23Nov22======================================================================= +1. The BadWord system has been rewritten. This change is pretty radical and + needs to be used with care as a word that is entered will be reduced to the + minimum sized string needed to match that word. + + This is effective a sysop command changeable version of the file badw_regex + but in a much more sysop friendly form. The (un)set/badword ... + commands now update the /spider/local_data/badword.new file in real time + without having to mess about with editing files and running load/badword. + load/badword still works, but you should now never need it. +19Nov22======================================================================= +1. "Fix" Badword detection in spots and announces etc. + NOTE: setting $DXCommandmode::maxbadcount to 0 (default 3) will disable + autologout on swearing. But any (detected) swearing will be logged. + User generated swearing will not be propagated, not will incoming + protocol. +2. Fix is_ipaddr so that it accepts more (most?) valid IPV6 addresses that + actual real IP addresses that might appear as endpoints. +3. Add CTY-3237 prefixes. +06Jun22======================================================================= +1. Fix show/wcy so that it works even if no wcy broadcasts have been received + this month. +01Jun22======================================================================= +1. Fix show/(v)hfstats commands. +2. Add CTY-3221 Prefixes. +07Feb22======================================================================= +1. (re)add set/seeme and show/seeme commands which don't appear to have + "taken". +03Feb22======================================================================= +1. Improve/add the help text for grepdbg and watchdbg. +31Jan22======================================================================= +1. Set $RBN::respottime to 3 minutes. NOTE change of name from + $RBN::minspottime. +2. Allow callsigns connected to a node with an SSID to get SEEME RBN spots + for their base callsigns. So G1TLH-2, G1TLH-7 will get **SEEME** RBN spots + if G1TLH is spotted on the RBN system - even is are both connected to the + node and G1TLH is not. Each call is unique and each callsign (variation) + must have done a set/seeme + set/skimmer. . +29Jan22======================================================================= +1. Implement RBN set/seeme which displays any passing RBN spots for your + callsign in "raw" format. +28Jan22======================================================================= +1. Add Capabilities Line to logged in users. +2. Make absolutely sure that all DB_Files are closed correctly. +3. Introduce (un)set/debug rbnchan to control the visualisation of raw RBN + input lines. +25Jan22======================================================================= +1. Fixed grepdbg so that it does what -help says it does. +24Jan22======================================================================= +1. Change Local::pcprot() calling conventions to be the same as the rest of + software (see any 'sub handle_nn' function in DXProtHandle.pm. +2. Corrected some interpolations in Messages. +3. Tried to make users file deletes and updates more stable. +22Jan22======================================================================= +1. Refine spot display to retain the ':' and also reduce the likelyhood of + of misalignment by using the available space in the 39 characters more + intelligently. This is WIP. +21Jan22======================================================================= +1. Fix version tracking in PC92 +2. Bring the major parts of master and mojo together to make maintenance a + bit less challenging. This does add anything extra to mojo, but it's still + quite a bit of work. 18Jan22======================================================================= 1. fix (un)set/registered. 09Jan22=======================================================================