OpenCPN - OpenPlotter - KPlex - RPi - CPU Maxed Out

[Dsanduril]

There are too many pieces to this puzzle, so asking the question here in case there are others with similar experience, but no idea yet if the problem is OpenCPN or one of the other numerous software layers. Would also welcome any and all troubleshooting tips to try and sort out that part of the question.

I have a working OpenCPN/OpenPlotter installation on a PI 3. Have had it running for 2 years, with updates to 4.8.6/1.2.0 (updates run every couple of months, last one today). It does 99% of what I want (giant thanks to the people that have put in the time and effort to get it there).

What works:

I have an AIS the provides data to the PI through a USB connection. Programmatically the AIS is connected to Kplex. Kplex then has a default TCP output that I have configured in OpenCPN as an input so that O' can receive the AIS and position information. SignalK also receives the same data from the same place.

Also attached to the PI is a USB/serial converter that feeds NMEA data to my autopilot. Programmatically the USB/serial adapter is connected directly to OpenCPN, bypassing Kplex, etc. (there are reasons I ended up here).

This all works great. There are a few little hiccups with the NMEA output data from time to time, but those are pretty minor and are probably feature requests more than problems. When this is all working the PI runs quite nicely at about 8% CPU usage. Everything is hunky-dory. SignalK shows all the AIS data, O' receives it all, and everything processes nicely.

What doesn't work:

I use O' to plan my routes and then serve up waypoint and course information to the AP. What I would like to do is also have that data in SignalK so that I can display the XTE/COG/etc. on SignalK displays.

To do that I add a network connection in OpenCPN to output to localhost. And then add a network listener in KPlex. I've tried both TCP and UDP, and localhost, 127.0.0.1, and 192.168.0.4 (the local LAN IP of the PI). I've also tried moving the serial output from OpenCPN to Kplex so that there is no duplication (so the output data would go O' to Kplex to AP and SignalK).

When I activate both the output in OpenCPN and the listener in KPlex the PI CPU usage runs up to the high 80s/low 90s % and the system becomes basically unresponsive. Doesn't seem to matter which of the various configuration options I try - when I have a network output in OpenCPN feeding a network input in Kplex I get an unusable system. There have been times I thought it occurred only when I had an active route in O', but I'm pretty sure at this point that it happens even with no active route.

So, there's the long-winded question - any ideas on what could be driving a huge jump in CPU usage when outputting from O' to Kplex over a local connection?

We're waiting on weather at the moment, but could start a passage any time in the next couple of days if the weather looks nice, so if I don't respond for a week or so it's not because I asked and ran away, but because I don't have internet at sea, will definitely look again when we get in.

[Dsanduril]

So, I've been staring at/working on this problem for six months, sometimes typing it out helps.

After posting I realized I was probably creating a loop in the NMEA data, and indeed the NMEA debug window showed that is the case. Why haven't I noticed that any of the other times I spent hours staring at the debug streams? Anyway, by putting filters on the EC data I was able to break the loop and everything now seems to function properly. Posting so a) people don't waste time, and b) in case someone else has a problem that leads them to this thread. With networking and multiple cooks in the kitchen it's all together too easy to create an NMEA data loop and drive CPU usage through the roof.

[Hakan]

Dsanduril..
If it helps you're not the first one creating this loop. There are also a couple of warnings in the wiki when descriptions for network transmit are described.

Good you eventually found the solution and thanks for the info.

[Dsanduril]

What's really sad is I should have known better I've even advised a few posters here when they described loops and yet I didn't see this one. Oh well, live and learn. I think I still have a little bit of fine-tuning to do on the filters, bit otherwise all is good.

[conachair]

Tiny unrelated point I didn't know 'til recently - NMEA input to Opencpn actually comes from Signalk on tcp port 10110 now, v1.0.0 anyway ..

[seandepagnier]

It is very easy to create data loops if you have multiple devices running opencpn. Maybe we should detect this and make a warning?

The opencpn databus is also very inefficient.

[Dsanduril]

Originally Posted by boat_alexandra

...The opencpn databus is also very inefficient.

Wish I had the skills to change that

In a more general discussion, one observation is that our AIS provides data at 38,400 and position updates at 1Hz (was 5Hz). It seems this is more than OpenCPN can deal with on the output side at 4,800 baud. Not sure why, as it seems that with AIS sentences dropped there is room at 4.8k to send all the position sentences along with the APB related sentences (calculated by O'). What I see is that the serial output goes fine, then there is a period where all the sentences show up in the debug window as dropped, then it all goes fine, then there's another drop, then ..... Maybe that's part of your "inefficiency"?

It all works for me, but every time I see the dropped block of sentences I wish I could tell the system to drop selectively (rather than "bus full" or however else the decision is made). Even better would be to be able to average a noisy input and output at a specified frequency.

That's definitely a new feature and since I'm pretty sure there are plenty of other things people would like...

While I'm here, and once again, thanks to all the people who have made it happen (and who then also come here to discuss and help with problems). I like having a system where I'm not locked into a specific manufacturer's decisions and hardware and greatly appreciate that one is available (even if from time to time I think there are improvements that could be made - that's pretty much true of any software or boat).

[seandepagnier]

It drops packets if it cannot keep up (recursive event loop) this is a known issue. The inefficiency is the fact it cannot keep up with typical datastreams which should not be very difficult. It is the way the databus is implemented. In my opinion it is a bug that it drops the packet.

As for dealing with slow serial.. why are you sending AIS messages on a 4800 baud serial connection? This is not normal.

[Dsanduril]

Guess I wasn't clear, I drop the AIS messages, then send the position/SOG/COG from the AIS GPS and the APB/related info from the OpenCPN on the 4800 serial connection.