UDP reuse?

[rsl360]

I receive my position and heading for OpenCPN via UDP. (UDP is awsome! And underutilized.) I also use these same broadcasts for other programs, or I would if I could. Once OpenCPN is running, the ports are no longer available to any other programs. I think you might be able to use the ReuseAddress switch in your code to get around this.

I would simply turn off the ports for a few moments, but that does not work. OpenCPN does not release the ports until it is shut down.

Thanks

Robbie

[bcn]

Robbie,

who is doing the UDP broadcast?
And which version of OpenCPN you are running?
For the documentation of the beta series 3.3.nnnn have a look here (if you haven't done already):
Data Connections | Official OpenCPN Homepage

We have one system that is broadcasting the GPS and AIS data via UDP and can connect and disconnect as many (tried up to four) systems running OpenCPN. All >= 3.3.1606. No problem.

bcn

[rsl360]

Hi

>who is doing the UDP broadcast?
Serial output of a GPS goes to a Nport Moxa 5410 on the network.

>And which version of OpenCPN you are running?
Version 3.3.1419 Build 2014-02-19

I've gone through the instructions again, they don't seem to have the answer I'm looking for. But I have learned how to disable my connections from the .conf file, which is handy if you hose the connections.

I've gone over this a bit more and I think I should give you a bit more background on my existing setup. I'm on a scientific research vessel, our network is probably a bit more complicated than most. But the main issue seems to be this: Our UDP broadcasts are already on predefined ports. I cannot change them. So GPS comes in on port 55000, and heading comes in on 55100. On my personal laptop, I have (other) programs that use those ports. Once I start OpenCPN, I no longer can access those ports on my laptop. I've tried outputting the data via OpenCPN, on the same ports, but it does not seem to work. I'm pretty sure I'm outputting the data properly, as it shows up in the NMEA debug window, but my other programs are still blocked.

We ran into this same problem with some of our own programs a few years ago. While perl is not C, we made this change to one of our programs to fix this:
Old was this way, it blocks-
$sock = IO::Socket::INET->new(Proto => 'udp', LocalPort => $ARGV[0]);
Changed it to this, it does not block.
$sock = IO::Socket::INET->new(Proto => 'udp', LocalPort => $ARGV[0], Reuse=>1);
I kind of think of this as the network version of opening a file in read/write mode, versus read only.

Robbie

[bcn]

Robbie,

can you try the latest beta version (3.3.1825)? In the lat months some improvements have been implemented with regards to the network coms.
UDP as a connection-less protocol should not block at all from the listener side.

Hubert

[muttnik]

I believe I understand what Robbie is asking and that particular situation isn't changed in the latest beta. It's a really good question, but the answer may be complicated by the way OpenCPN does UDP sockets and *might* (research/community input required) have cross platform differences.

The problem is that normally speaking you can't bind two sockets with the same local address/port pair. If two applications try it, the second will fail. Robbie would like to have more than one process bind a UDP socket with the same address/port pair (OpenCPN and one or more other applications) with each socket getting a copy of any broadcast (or multicast) packet received. He's achieved this in his perl program using what I presume maps onto the SO_REUSEADDR option which has a rather different effect for UDP sockets than it does for TCP.

OpenCPN doesn't bind UDP sockets to a particular broadcast address, it binds to the wildcard address (INADDR_ANY) so will receive broadcast *and* unicast packets to the specified port on any interface.

On OpenCPN's various supported platforms, SO_REUSEADDR and SO_REUSEPORT are in the frame for being part of the solution but we should also probably consider what happens to unicast datagrams fired at the port: behaviour may well be different than from multicast/broadcast. Personally I don't know the answer and in stuff like this I've learned that sometimes man pages lie, so experimentation may be needed (and any practical experience would be of interest).

As for workarounds, the ports used by the original broadcaster may be fixed but are the ports used by the other programs on your laptop? If not you could read the original broadcasts in OpenCPN then re-broadcast on another port (and maybe on the loopback interface?) which your other applications could then read. The latest beta of OpenCPN is useful here as you can specify output only for UDP, but you can do it with earlier versions and priorities.

What platform are you running on and can your other apps read from a serial port instead of the network?

[bcn]

Originally Posted by muttnik

I believe I understand what Robbie is asking and that particular situation isn't changed in the latest beta. It's a really good question, but the answer may be complicated by the way OpenCPN does UDP sockets and *might* (research/community input required) have cross platform differences.

The problem is that normally speaking you can't bind two sockets with the same local address/port pair. If two applications try it, the second will fail.

Didn't catch that the situation is "various processes on the same machine".
So a similar case as trying to access the same COM port twice...
Need for an internal dispatcher.

[PjotrC]

Originally Posted by bcn

try the latest beta version (3.3.1825)?

Didn't try this yet, but indeed 3.3.1731 is much better in this area than 3.2.2.

I use UDP between various FlightGear programs, my naively written Python modules, and OCPN. FlightGear and Python always connect immediately, while OCPN sometimes needs restarting all systems...

[muttnik]

A little research and experimentation later...

The short non-technical version is that we can implement what rsl360 suggests (possibly with some minor code scruffiness to accommodate cross platform differences) and it will allow multiple applications on a single computer to listen to the same broadcasts if they all play nice together. However if something is firing packets directly at our computer (an edge case, but I'm pretty sure I've seen reports of people using devices which do this for NMEA-0183 over IP on boats) only one socket will receive each packet. Which one (first to start, last to start or random) is dependent on platform. That bit I'd say was undesirable: You want it to work or tell you it's not going to work, but the undesirable behaviour might be outweighed by the benefits: a good discussion point I think. Thoughts anyone?

The short technical version is that we need to use SO_REUSEADDR or SO_REUSEPORT depending on the platform. If we bind to INADDR_ANY, multicast and broadcast packets are delivered to all sockets bound to a given port and unicast are delivered to only one. On OS X Mavericks this seems to be the first socket to bind. On Ubuntu 12.04 (using SO_REUSEADDR as this was before SO_REUSEPORT was implemented) it's the last socket to bind. What I've read of SO_REUSEPORT in the 3.9 and later kernels suggests that unicast packets will be divided evenly between receiving sockets. Whether the semantics of SO_REUSEADDR have changed in Linux since SO_REUSEPORT was implemented, and what happens to unicast if both are in force I'll have to install a newer Linux system to see (unless someone who knows can tell me first as my new sails are likely to receive more attention this weekend than my keyboard). I shamefully confess to not having a windows development environment but technet tells me that SO_REUSEADDR should do what we want for broadcast and multicast, although not which of multiple receiving sockets gets unicast packets. Not that it matters...

There's a longer technical version too but it's past my bedtime :-)

[rsl360]

Originally Posted by bcn

Robbie,

can you try the latest beta version (3.3.1825)? In the lat months some improvements have been implemented with regards to the network coms.
UDP as a connection-less protocol should not block at all from the listener side.

Hubert

I installed
Version 3.3.1824 Build 2014-06-24 (1824 is what I got using git, not sure where 1825 is)
I'm running solydXK, which is a debian derivative similar to ubuntu.

Short answer, nothing really changed. I tried a few combinations. I turned on output for the input ports, no change. I created separate ports to output the pos/heading inputs, on the same ports, this make a big mess and eventually uses up all my memory. (I guess I'm not surprised at that.) I also tried to output the data on a different port. (receive on 55000, send back out on 55001) This also did not work. I was surprised by this at first, but then I realized that it may be the same issue. Once OpenCPN opens the port, (to broadcast), my other programs can no longer open it. Basically, once OpenCPN opens a port, nothing else can get that port.

After reading all the other comments, I may need to rethink this.

Robbie

[nohal]

Robbie...
There is no 1825 (the 1825 deb is just a version bump to fool Launchpad as I needed to include a 2 line fix to the build process and was lazy to produce a proper patch... Code-wise it is 1824)
The problem and solution was pretty well outlined by muttnik, now just someone has to find the time to implement it. Not me in the coming weeks - I already have a few too many unfinished tasks piled up on the table.

Pavel

[muttnik]

I'll have to experiment with this. I note that the code already claims to set the wxwidgets flag which I believe is supposed to set SO_REUSEADDR. If this was MacOS the explanation would have been in my previous post, but you're using Linux so when I get time I'll try and work out if this is a wxWidgets issue.

Meantime if you're using the latest beta and you're trying re-broadcasting on another port, have you tried making the re-broadcasting datastream *output only*? You'll notice a (new) checkbox next to the "output on this port" one which says "receive on this port". Uncheck that and make sure output is checked. That way Opencpn should not bind a socket on that port so your other progs should be fine.

[muttnik]

ok...just tried this on ubuntu (12.04): udp data connection in OpenCPN. kplex (which sets so_reuseaddr) successfully started and listening on the same port. Both received the same broadcast sentence. You absolutely certain that these other programs you're trying to use are setting REUSEADDR?

[rsl360]

Originally Posted by muttnik

Meantime if you're using the latest beta and you're trying re-broadcasting on another port, have you tried making the re-broadcasting datastream *output only*? You'll notice a (new) checkbox next to the "output on this port" one which says "receive on this port". Uncheck that and make sure output is checked. That way Opencpn should not bind a socket on that port so your other progs should be fine.

I had not tried that, but now I have and it works. I can rebroadcast my incoming UDP string on a different port. I then change the ports on the other software to receive the data on the new port. This is workable for me.

It took me a while to figure out how to send out data with bringing it "in". That threw me off for a bit. But now that I understand the giant bit bucket that all the data circulates in, it's a bit clearer.

Robbie