Preventing data loops between ShipPlotter and OpenCPN ?

[bigbloke]

Hi All,

I have RTFM'd , I have searched the forum (admittedly not to exhaustion)
so here goes with a question.

An "Internet neighbour" and I wish to shared UDP AIS data between ourselves
in order to improve our own non-commercial AIS coverage as well as offering our data to a number of online services , such as shiplotter, marinetraffic etc.

I cannot run Shipplotter on my PC as I have Planeplotter and several other professional apps that prevent co-exstence with Shipplotter. I therefore use
OpenCPN and accept that I only receive local area data (which is all I really want)

our "sharing goals" are simple enough:

* My Neighbour would like to see traffic from Newport locally in Shipplotter (He can see the majority of ships, but not those in parts of Newport Docks and Portbury)

* I would like to see Cardiff locally (I have blackspots created by two islands in the river Severn and a local obstacle)

* I would like my neighbour to be able to share my Newport data with Shipplotter and Shipfinder (I expect no privilege from either site in return)

* I do not want to see global content coming in from shipplotter on my OpenCPN PC (nor should I as I am not a direct contributor)

* We both would like to continue contributing to marinetraffic.com but should either of us go offline, we would like the other's feed to continue

So we set up a test harness (as per the attachment)



And whilst we could indeed see each others data, it created an AIS feedback loop between our respective sites resulting in 10s of thousands of messages a minute. We have therefore suspended testing.

I have read about input and output filters in OpenCPN and freely admit that the AIS payload is not my expertise (im more a firewalls, VoIP, SOAP , SQL, routers, and 3GPP protocols kind of person as my day job) But is there a way to filter UDP output traffic based upon connection source please ? I.e. can I say only traffic that came from my COM1 can go out over the UDP connection to my neighbour ?

I have set my own com 1 priority to 9, my primary output (marinetraffic) to to 8 and forwarding to my friends PC as everything between 0-7 which has made no difference to the message storm. I have asked a similar question to the developer of Shipplotter by email. Is there some way to "Tag" my outbound traffic with talker ID then remove it when it comes back around the loop with an input filter to prevent it continually going around ?

Grateful for any assistance you may be able to offer

Many thanks in advance

Regards

BB

[conachair]

One option might be to use node-red and mqtt. *Should*... be quite easy to set up once you get acquainted with node-red - https://nodered.org/

Possible solution - Deselect the serial ais connection in opencpn, then in node red make a serial connection input which goes to a MQTT output with, say, topic "My_AIS", also as an output make a UDP or TCP then set OpenCPN to look at this connection. Neighbour does the same sending to MQTT topic "his_AIS", then you make a MQTT input looking at topic "his_AIS" also connected to the same output as before. That way he can see your AIS nasa AIS and you can see his oevr the internet. Plus MQTT is really frugal with the amount of data it uses so you don't hit the limit if using mobile data.

[muttnik]

Caveats: It's been a while since I looked at the OCPN code in depth so my info could be outdated (any feel free to point out errors). Plus I have no experience of ship plotter. I'm *assuming* (and a 5 min scan of the shipfinder site hasn't clarified this) that the cloud in the diagram labeled "shipfinder" is an Internet feed from shipfinder which is in some way associated to the subscription to shipplotter.

To start with a direct answer: no. I don't think you can do what you want using only changes to OCPN setup. Setting priority is only going to exclude lower priority AIS inputs and you actually do want all the inputs.

No there's no way (or didn't used to be) of filtering outputs based on which input it came in on.

Yes there is an NMEA standard way to identify sources of data for filtering purposes (NMEA-0183v4 source TAGs) but you'd be hard pressed to find consumer software which does much with that. OCPN just discards any v4 TAGs.

You could solve most of the problem with a mac or linux box (e.g. raspberry pi) running kplex which *can* filter outputs depending on which input it came from (I can go into details if you're interested). The problem this would leave out would be that the feed from your pal is still a combination of AIS you don't want (from ship finder) and AIS you do want (from your pal's AIS). That's something that really only your mate can solve. Presumably ship plotter can distinguish between sources, otherwise you'd be sending all your ship finder data to ship plotter, which is presumably also a data loop.

If you don't want to bring an unfamiliar OS into things, I'd probably be looking at the capabilities of Windows-based multiplexor software.

Question though (I may have misunderstood...). It *seems* as though you want to send each other's data to Marine traffic if their link goes down, but if the Interweb fails for one of you, how will you contact the other, and if MarineTraffic itself is unavailable, neither of you will be able to send anyway.

[Paul Elliott]

You can use the NMEA router feature of NavMonPc to block data loops and otherwise send data where it needs to go. The NavMonPc router will direct NMEA0183 sentences to particular output ports based on input port and sentence header.

[Moo]

Use a some multiplexer software I use GPSGate's client currently called Splitter I think. GpsGate - download

With that you can filter only sentences and to from each virtual com port or TCP or UDP connection. You can use the identifiers in the filters.

You could also use the GPS sentence plug in to change identifiers to help with the filtering.

I run inputs into my boat computer from several ports USB, serial, and network, mulitplex it all in GPSGate Splitter and deliver it to OCPN, TZ, OziExplorer, Airmail, WXSat, and log it all to hourly files. I have a few other virtual com ports set up for some of the apps that I use less frequently.

[rgleason]

"the GPS sentence plug in to change identifiers to help with the filtering."
NmeaConverter_pi?

[Moo]

Originally Posted by rgleason

"the GPS sentence plug in to change identifiers to help with the filtering."
NmeaConverter_pi?

That's the one

[rgleason]

I like the navmonpc solution too.

[conachair]

Just set up node red to output mqtt on a Pi and a laptop to receive so the AIS data goes over the internet.

Took longer to record than to set up Though some of the work was done already, all AIS gets saved to a file. Just cos it can

[bigbloke]

Thanks for the replies all,

I looked at the various solutions and went along Moo's suggestion (thanks!)
it has worked perfectly - and has given me some additional ideas of better ways to support serving data.

For the benefit of those following in my footsteps in the future, here's how it works out.

(With reference to the attached diagram)

Data comes out of the Nasa AIS3 and is pumped into GPSGate express splitter - which is the freeware version of GPSGate.

GPSGate runs as a small "tray" app and is a data splitter that can support outputs to virtual com ports and TCP or UDP data connections etc.

In the GPSGate input tab I selected COM 1
In the output tab I selected virtual com 3 and the IP address of my neighbour as a UDP target.

That was it for GPSGate Splitter

I then setup OpenCPN:

virtual COM3 IN as priority 9
I set UDP data IN from my neighbour as priority 8
my output to marinetraffic.com as priority 7
all my onsite UDP outs to Priority 1

to counteract the latency between our two sites I added a 3 second delay filter on input priorities and hit the apply button.

My neighbour received my data for 11 hours without a hitch

He then re-enabled sharing with me which I port-forwarded directly into OpenCPN thereby breaking the data loop (opencpn never sends data back to him) we both transmit our combined data up to the internet sharing sites.

Were averaging about 200 messages a minute unlike the 10000 when we had the loop !! (its a good stress test for any plotting PC :-)) and achieve a
combined coverage of 600-800+Sq KM from the Road Bridges in the River Severn down towards Ilfracombe. Not too shabby at all.

Many thanks once again to all for the helpful suggestions.

Regards

BB

[bigbloke]

the missing diagram:

[Moo]

Originally Posted by conachair

Just set up node red to output mqtt on a Pi and a laptop to receive so the AIS data goes over the internet.

Took longer to record than to set up Though some of the work was done already, all AIS gets saved to a file. Just cos it can

Nice demo but it is all happening too fast for me. How do you pause a gif?