Rpi Developer says "slimm(ing) down your application". How ?

[bdbcat]

Chris...


Send me the logfile. I'll try it.
Dave

[NahanniV]

Originally Posted by bdbcat

Hello all...


More testing here.
Rpi 3, 1GB main memory, "official" 7" touch screen.

Fake GL driver
Running long-ish VDR playback of a trip over which I have RNCs available. Running continuously, so looping back to restart over and over.
Raster Chart Zoom factor bumped up to "4".
Tracking turned on.
Running "top" to monitor memory use.



Results:

a). Memory reported used by OCPN rises to about 20%, and stabilizes.
b). OCPN CPU load steady about 30%
c). Allowed to run 2 hours. Stopped VDR playback. Normal operation unaffected. Zoom/Pan OK.


Next steps:
We really need a VDR recording of a representative failure case. Any one got a long VDR recording of, say, US ICW trip north, at high resolution?


Thanks
Dave

Hi Dave,

What did you have your GPUTextureMemSize set at during your tests ?
I have found that setting it above 128 will eventually cause a crash, sometimes a segmentation fault, sometimes just hanging. The system log reports VC4 not getting CMA memory, just like what I have seen. If I set it to 256 this happens quite quickly.

What distribution are you using, and what is CMA memory set at ?
CMA memory seems to be set when compiling the Kernel.

We will be leaving Georgetown Exumas in the next week, and will try to record everything. Eventually we will be back to US charts.

A couple of questions:
What do you mean by a high resoloution recording ? Is there a setting for this in VDR ?
Will the file size become too big at some point ?
If the machine recording with VDR crashes, will the data recorded up to that point be saved ?

[bdbcat]

NahanniV...


I am running raspbian, Stretch. Not sure about CMA memory setting. How do I find this out? It would be good to make sure that we are on exactly the same configuration.


I'll try to increase texture memory allocation to provoke the fault sooner.



There are no settings for VDR resolution. He records what comes in, without modification. I was referring to the possibility of converting an old high-res track into a vdr stream. On linux, vdr data is usually saved on crash, but not always.

Also, can you post a screenshot of "top" while running OCPN, just at anchor with live data?
Finally, what is your screen resolution? I am running on a small screen. More pixels = more memory....



Thanks
Dave

[transmitterdan]

Dave,

Graphics memory is partitioned in /boot/config.txt

[bdbcat]

Dan...


OK, I have tried fiddling with that. No detectable change in behavior, or "top" memory reports.
Now running previous test with OCPN texture memory set to 256. We shall see...


Thanks
Dave

[NahanniV]

Originally Posted by bdbcat

NahanniV...

I am running raspbian, Stretch. Not sure about CMA memory setting. How do I find this out? It would be good to make sure that we are on exactly the same configuration.

I am running the OpenPlotter distribution Updated to OpenCPN5 and the latest 4.19.y Kernel, I think that's Raspian Strech as well, not sure if there are any modifications.

I see CMA memory allocation of 256M in the system log:

pi@openplotter:~ $ dmesg | grep cma
[    0.000000] cma: Reserved 256 MiB at 0x1ec00000
[    0.000000] Kernel command line: 8250.nr_uarts=0 cma=256M bcm2708_fb.fbwidth=1024 bcm2708_fb.fbheight=768 bcm2708_fb.fbdepth=16 bcm2708_fb.fbswap=1 vc_mem.mem_base=0x3f000000 vc_mem.mem_size=0x3f600000  dwc_otg.lpm_enable=0 console=ttyS0,115200 console=tty1 root=/dev/mmcblk0p7 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
[    0.000000] Memory: 735368K/1021952K available (8192K kernel code, 627K rwdata, 2164K rodata, 1024K init, 821K bss, 24440K reserved, 262144K cma-reserved)
[    3.617353] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[    3.619204] bcm2835_vc_sm_cma_probe: Videocore shared memory driver
[    3.636565] vc_sm_cma_vchi_init: failed to open VCHI service (-1)
pi@openplotter:~ $

Originally Posted by bdbcat

....
Also, can you post a screenshot of "top" while running OCPN, just at anchor with live data?

top - 10:38:52 up 4 days, 11:55,  1 user,  load average: 0.81, 0.81, 0.77
Tasks: 143 total,   1 running,  94 sleeping,   0 stopped,   0 zombie
%Cpu(s): 13.4 us,  2.8 sy,  0.0 ni, 83.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :   998536 total,   144088 free,   529152 used,   325296 buff/cache
KiB Swap:  1023996 total,   958460 free,    65536 used.   358468 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND     
25687 pi        20   0  496040 259784  58604 S  21.6 26.0 309:09.11 opencpn     
  585 root      20   0  114848  26040  17688 S  13.8  2.6 759:30.42 Xorg        
 1014 pi        20   0   54072   8604   4168 S   6.2  0.9 339:39.03 python      
  553 root      20   0   43856  18660   9860 S   5.2  1.9   1:37.83 vncserver-+ 
  423 pi        20   0  158828  60312   8360 S   4.6  6.0 275:29.93 node        
 2645 pi        20   0   48208  18956  15768 S   1.6  1.9   0:01.30 lxterminal  
 2657 pi        20   0    8404   3088   2640 R   1.6  0.3   0:01.15 top         
 1024 pi        20   0   35388  16492   3216 S   1.3  1.7  80:33.13 pypilot_bo+ 
  647 root      20   0   11900   6832   6488 S   1.0  0.7   0:05.43 vncagent    
 1056 pi        20   0   35392  15780   2496 S   1.0  1.6  35:45.80 pypilot_bo+ 
  369 root      20   0   13364   1552   1520 S   0.7  0.2  47:17.16 eGTouchD    
 1058 pi        -3   0   35388  15980   2708 S   0.7  1.6  37:49.14 pypilot_bo+ 
 2336 root      20   0       0      0      0 I   0.7  0.0

Originally Posted by bdbcat

Finally, what is your screen resolution? I am running on a small screen. More pixels = more memory....

1024 * 768 resistive touchscreen, with "TouchScreen Interface" enabled.

One more question for you:
If I have a crash, is there any information I could collect that might be useful ?
Even if the display is hung up, I can usually use ssh from another computer to view things.
dmesg: always seems to be similar.
How about glxinfo ?

[NahanniV]

Originally Posted by bdbcat

Dan...


OK, I have tried fiddling with that. No detectable change in behavior, or "top" memory reports.
Now running previous test with OCPN texture memory set to 256. We shall see...


Thanks
Dave

I spoke with the developer of the new VC4 driver, and he told me that his driver is not using that memory, and that it is wasted.

I have it set at 16M. and see no difference between that and configuring it at 256M.

[bdbcat]

NahanniV...;


"and that it is wasted."
Can that be recovered in any useful way?
Dave

[verkerkbr]

Originally Posted by NahanniV

I spoke with the developer of the new VC4 driver, and he told me that his driver is not using that memory, and that it is wasted.

I have it set at 16M. and see no difference between that and configuring it at 256M.

That is right you see no differences

VC4 driver uses mainly the graphic processor. Therefore it is of no use to change the video memory. I get the best result with the setting on 64.

And if you enlarge the video memory, the allocation of this memory part can probable not be used for other processes.

The RPI3B+ with OpenPlotter is working here without problems with latest kernel version 4.19.37-V7+.

But on Oesenc charts and CM93.

Regards,


Bram

[NahanniV]

Originally Posted by bdbcat

NahanniV...;


"and that it is wasted."
Can that be recovered in any useful way?
Dave

I don't know how the VC4 driver works, but it is clear from his email that the new driver is not using the previous GPU memory split configuration.

It is using memory allocated from Contiguous Memory Allocation.

I asked the developer of the VC4 driver for clarification:

> Thanks Eric,
>
> I will speak with the developer.
>
> Can you also please give a definitive answer as to the use of "Memory
> Split" or "GPU Memory" configuration when using your experimental driver.
> Is it used at all when your driver is invoked ?
> Can It be set to minimum 16M ?
> There is quite a bit of confusion on that subject.

gpu_mem is not used by vc4, so whatever the firmware isn't using is
wasted.

I would have liked a clearer explanation than that, but did not pursue it further.

Anyway in my testing when the new VC4 driver is enabled, I see no difference if the Rpi GPU memory is configured for 16M or 256M.
So I leave it a 16M.

[bdbcat]

NahanniV....


Testing some more.
There is a config file parameter that suggests to OCPN how much main memory to use before more aggressively attempting to free any un-used memory. The factor defaults to one half of available RAM seen after OCPN is started and initialized. For a 1 GB Pi, this will be about 400 MB. This default works fine for reasonable memory sizes as found on laptops and desktops, for example. Also good for systems with independent GPU memory resources.



However, for systems with shared graphics memory and limited total resources (like rPI), the runtime logic tends to undercount the memory dynamically used by OCPN, especially when using RNCs without texture compression.


So you may have some better results by setting this factor manually in the config file.
Try this:

[Settings]
MEMCacheLimit=200

I don't see much difference in performance casually. I can definitely see the positive results in "top" as I pan/zoom around a raster chart set.



Good Luck
Dave

[NahanniV]

Originally Posted by bdbcat

NahanniV....


Testing some more.
There is a config file parameter that suggests to OCPN how much main memory to use before more aggressively attempting to free any un-used memory. The factor defaults to one half of available RAM seen after OCPN is started and initialized. For a 1 GB Pi, this will be about 400 MB. This default works fine for reasonable memory sizes as found on laptops and desktops, for example. Also good for systems with independent GPU memory resources.



However, for systems with shared graphics memory and limited total resources (like rPI), the runtime logic tends to undercount the memory dynamically used by OCPN, especially when using RNCs without texture compression.


So you may have some better results by setting this factor manually in the config file.
Try this:

[Settings]
MEMCacheLimit=200

I don't see much difference in performance casually. I can definitely see the positive results in "top" as I pan/zoom around a raster chart set.



Good Luck
Dave

Ok, will set MEMCacheLimit, but I don't think I'm running out of conventional memory. I never see an indication of swap memory having been used when I run HTOP.

In adding MEMCacheLimit to my config file, I noticed that
GPUTextureMemSize was set back to 128. I had set that to 64 on your advice. I tried to set it back to 64, but it seems to be written back as 128 ?

[bdbcat]

NahanniV...


The code forces GPUTextureMemSize to be 128MB or more.


Conventional memory and GPU memory are coming from the same pool.

Are you sure you have swap running? I see swap activity when I zoom and pan aggressively.. This is probably the main reason why the system gets so sluggish under memory pressure.
I recommend to disable swap for testing, and probably for real use when reliability counts. It is pretty easy to trash an sd-card if the system crashes with swap active....



Dave

[NahanniV]

I have been running with:

MEMCacheLimit=200

For weeks at a time without problems.

That is almost exclusively with scanned raster charts in the Bahamas.

I recorded a few VDR data files, but they are very large.

I am now back on the east coast of the USA and will be interested to see what happens with a mix of the official US ENC and RNC charts as we do some passages inside on the ICW.

Thanks for all the help with this.

[NahanniV]

Originally Posted by bdbcat

NahanniV....


Testing some more.
There is a config file parameter that suggests to OCPN how much main memory to use before more aggressively attempting to free any un-used memory. The factor defaults to one half of available RAM seen after OCPN is started and initialized. For a 1 GB Pi, this will be about 400 MB. This default works fine for reasonable memory sizes as found on laptops and desktops, for example. Also good for systems with independent GPU memory resources.



However, for systems with shared graphics memory and limited total resources (like rPI), the runtime logic tends to undercount the memory dynamically used by OCPN, especially when using RNCs without texture compression.


So you may have some better results by setting this factor manually in the config file.
Try this:

[Settings]
MEMCacheLimit=200

I don't see much difference in performance casually. I can definitely see the positive results in "top" as I pan/zoom around a raster chart set.



Good Luck
Dave

Could this be changed to automatically work better on Rpi ?