Opencpn (on Ubuntu bionic) crashes on startup when displaying S63 charts

[gnaffron]

Hi everyone,


I am currently running into trouble with my OpenCPN installation. I recently installed some s63 ENC charts (from AusENC) onto the system. The installation was fine (after a few false start) and I could pan and visualize the charts without issues. However when I close and the restart the program, the following happens:


Five processes called opencpn are started. One process uses 100% CPU on one core and keeps on allocating memory, until the system runs out and start swapping (that takes about 3 minutes). If not killed before that point the whole system freezes and there is nothing to do but a hard reboot.


The program itself is slightly responsive for about 10 seconds and then freezes completely. During that time it is possible to pan to an area not covered by the s63 chart and close the program. Upon startup it will then behave fine until faced once again with an s63 chart.


Here is my setup:
System: Ubuntu bionic x64
Opencpn 4.8.4
s63 plugin 1.8
oeSENC 2.0


I tried deleted the charts (rm -rf on the AusENC directory). That fixes the problem but for the fact that I don't have the charts I need). I also tried to disable the OpenGL acceleration, but this didn't change anything.


Anybody to help me?

[bcn]

Originally Posted by gnaffron

Five processes called opencpn are started. One process uses 100% CPU on one core and keeps on allocating memory, until the system runs out and start swapping (that takes about 3 minutes). If not killed before that point the whole system freezes and there is nothing to do but a hard reboot

....

Here is my setup:
System: Ubuntu bionic x64 (= 18.04 LTS)
Opencpn 4.8.4
s63 plugin 1.8
oeSENC 2.0

You did try to uninstall and re-install the plug-in I suppose?

OpenCPN shall run one daemon for S-63 and another one for oeSENC.
Running several instances of the S-63 service is usually not an issue though.

Can you please attach the OpenCPN log? (zip it and rename it to ...zip.doc to be able to upload it here)

(Something that does not solve your todays problem with S-63:
the complete Australian vector chart portfolio will be getting available under the oeSENC technology in short term)

Hubert

[bcn]

One more question:
does the crash - and the memory leak - come up always independently from the S-63 chart displayed?
Can you try to start at a different location with different charts on the canvas?
And from a view without S-63 charts displayed at all?
With "quilting" on/off, i.e. in single chart mode?

All this in order to identify if there is a specific chart causing havoc.

Hubert

[gnaffron]

Originally Posted by bcn

You did try to uninstall and re-install the plug-in I suppose?

Nope but I shall try it shortly (tonight when I come home)

Originally Posted by bcn

Can you please attach the OpenCPN log? (zip it and rename it to ...zip.doc to be able to upload it here)

Will do that tonight

Originally Posted by bcn

One more question:
does the crash - and the memory leak - come up always independently from the S-63 chart displayed?
Can you try to start at a different location with different charts on the canvas?
And from a view without S-63 charts displayed at all?
With "quilting" on/off, i.e. in single chart mode?

With no s63 chart displayed the crash doesn't occur. Also perfectly fine if a raster chart is displayed instead of the vector (s63) chart on an area covered by both.

I'll try to disable quilting and see what happens.

Thanks heaps for your help

[gnaffron]

Ah also, I am a software engineer, but mostly around Java-based techs. I tried my luck yesterday with gdb/ddd but to no avail on my system. To be honest I haven't done anything C-related since my uni days and rusty would be an understatement when describing my C/system skills.

However if you have a couple of pointers* to help me setting up a debugging env I'm happy to help investigate and who knows, maybe even contribute.

I guess the first step would be to build/install opencpn-4.8.4 from source with debugging symbol generation enabled, right?

* pun not intended

[gnaffron]

As requested here are some logs that might be of interest.


Both were generated with the plotter displaying the same area. The file opencpn.log.ok shows the case where a raster chart is displayed. In that case the program behaves fine. The second case shows when the program starts and display an s63 tile for the same area. Here one process goes berserk, it is impossible for instance to open the settings menu and the memory starts to be allocated at an alarming rate.


In both case I've disabled quilting. I just tried to uninstall/reinstall the plugin but I got the same behaviour

[bdbcat]

gnaffron...

I read the logfiles.

Please try to run this command, from a cli console:

/usr/bin/OCPNsenc  -n  -i "/home/antoine/Charts/AusENC/s63SENC/AU160150.es57" -o "/tmp/um8mVK" -u 1259E68C44F8711B8F7650893147 -e C4867B45 -b 49501 -p AU16015020181130FDCC45C8581A1F12B3AC5344771380A762FD2D2EB6E35F12,0,7,AU, -z "/usr/lib/opencpn/libs63_pi.so"

This line came from the logfile, and was the last thing executed before shutdown. Here OCPN is trying to extract the encrypted S63 chart cell and store the results (further encrypted) to local storage.

Thanks
Dave

[bcn]

AU160150 is a large cell - small scale - and very easy getting it "in the background" when zooming or panning in SE Australia

[gnaffron]

Originally Posted by bdbcat

gnaffron...

I read the logfiles.

Please try to run this command, from a cli console:

/usr/bin/OCPNsenc  -n  -i "/home/antoine/Charts/AusENC/s63SENC/AU160150.es57" -o "/tmp/um8mVK" -u 1259E68C44F8711B8F7650893147 -e C4867B45 -b 49501 -p AU16015020181130FDCC45C8581A1F12B3AC5344771380A762FD2D2EB6E35F12,0,7,AU, -z "/usr/lib/opencpn/libs63_pi.so"

This line came from the logfile, and was the last thing executed before shutdown. Here OCPN is trying to extract the encrypted S63 chart cell and store the results (further encrypted) to local storage.

Thanks
Dave

Here's the stdout from the command

$ /usr/bin/OCPNsenc  -n  -i "/home/antoine/Charts/AusENC/s63SENC/AU160150.es57" -o "/tmp/um8mVK" -u 1259E68C44F8711B8F7650893147 -e C4867B45 -b 49501 -p AU16015020181130FDCC45C8581A1F12B3AC5344771380A762FD2D2EB6E35F12,0,7,AU, -z "/usr/lib/opencpn/libs63_pi.so"
OCPNsenc: Could not open backchannel
OCPNsenc: Executing...
OCPNsenc: Creating eSENC Key...
OCPNsenc: Exit.

[bdbcat]

gnaffron....

Sorry I dropped the thread here. Busy....

Anyway, the CLI command I asked you to execute looks entirely correct for the environment. Thanks

So, lets come back to your original post:

You said:
"Five processes called opencpn are started. One process uses 100% CPU on one core and keeps on allocating memory, until the system runs out and start swapping (that takes about 3 minutes). If not killed before that point the whole system freezes and there is nothing to do but a hard reboot."

How did you see those processes start? Using ps, for instance?

Can you post some kind of screen shot so that I can see this in detail?

Next, we can try some gdb/ddd work, if you are game.

Simply:
0. Start OCPN, move to an area not containing S63. Set OCPN window to partial screen, to give room for ddd UI. Exit OCPN.

1. $ddd /usr/bin/opencpn

2. Program->Run (no arguments) Should start up like normal

3. Pan to an s63 area. OCPN goes runaway.

4. While OCPN is runaway, in ddd, do Program->Interrupt.

5. Do Status->Backtrace. Screenshot this.

6. Also maybe screenshot "ps -aux" on opencpn entries.

Dave

[gnaffron]

Hi Dave,


Thank you for coming back to this.

Originally Posted by bdbcat

How did you see those processes start? Using ps, for instance?

htop for that little bit extra fanciness...



I attached the ddd backtrace dump and a capture of htop, however I have the feeling that gdb is not picking up the sources it wants and the backtrace looks odd to me. Well, you'll be judge of that.


Let me know if I can provide any more information.


Thanks again so much for your time !


Cheers,
Antoine

[bdbcat]

Antoine...

Hello. Reviewing your logfiles again, let us try this:

Please remove/disable the "objsearch_pi" plugin.

Who knows....

For info:
When showing s63 charts. OpenCPN does not use a background daemon. All the offline work is done synchronously by a utility called "OCPNSenc", which runs to completion in a separate process as required.


Dave

[gnaffron]

just tried it, sadly to no avail.


I'm trying to sort out a decent C dev environment. I have the feeling that I miss quite a few packet to be able to debug properly. If you have any experience with gcd and ubuntu any help is welcome

[bdbcat]

gnaffron....
After some effort, this problem is now understood.

Root cause: wxWidgets 3.0.4 is shipped with Ubuntu 18.04, an upgrade from 3.0.2 shipped on previous Ubuntu releases.
There was some change in networking code deep down inside wxWidgets 3.0.4., which in-directly affected our Plugin. I changed some socket parameters in our interface, and so fixed it in a compatible way.

Now, the OpenCPN Production ppa shows s63_pi Version 1.12.1.
You may update your s63 Plugin to the new version, and you should be OK.

Please let me know how this goes.

Thanks for your patience.
Dave

[gnaffron]

It works like a charm! Thank you so much for that fix!


Tested with/without opengl acceleration, with/wo quilting, tested reimporting charts. No issue so far