Sean's Optimum Branch Merge

[sailorF54]

1715 up and running on W7...
Trivia: you must select English when installing, otherwise no text

* Selected at random (it's greek to me)
accelerated panning / texture compression / texture caching 64
for a low-end AMD Athlon Dual, 4 GB RAM, Radeon HD 3200 (with oGL 3.4, I think!)
Is that intelligent ?

* Plugin: although I have populated the plugin folder with those I have on 16.06 (exept grib and dashboard)
none (except your libgrib and libdashboard) are available
Typical log is

08:50:24: PlugInManager: Loading PlugIn: C:\Users\admini\Desktop\OpenCPN 1715\plugins\s63_pi\s63_pi.dll
08:50:24:  Error: Failed to load shared library 'C:\Users\admini\Desktop\OpenCPN  1715\plugins\s63_pi\s63_pi.dll' (error 127: la procédure spécifiée est  introuvable.)
08:50:24:    PlugInManager: Cannot load library: C:\Users\admini\Desktop\OpenCPN 1715\plugins\s63_pi\s63_pi.dll

So I am unable to test my S63 charts

* Ingested and displayed a 28 MB grib without a hitch

* Screen flickers when mousing over AIS target to display info, severe flickering when drawing routes and mousing over..

* Dashboard display broken (part of the text is written and slowly erased) (no influence of OpenGl params, even without)

* I'm not too sure about that: Both procs seem used with intensive task such as displaying the current markers on a small scale chart, but proc usage doesn't go over 50%

* Testing in progress...

[seandepagnier]

Originally Posted by sailorF54

1715 up and running on W7...
Trivia: you must select English when installing, otherwise no text

I am using a very old nsis installer program which has "broken" the language support in the installer, I am going to upgrade now.

* Selected at random (it's greek to me)
accelerated panning / texture compression / texture caching 64
for a low-end AMD Athlon Dual, 4 GB RAM, Radeon HD 3200 (with oGL 3.4, I think!)
Is that intelligent ?

To stay simple, the more boxes you check the more acceleration, but more potential for problems.

* Plugin: although I have populated the plugin folder with those I have on 16.06 (exept grib and dashboard)
none (except your libgrib and libdashboard) are available
Typical log is

08:50:24: PlugInManager: Loading PlugIn: C:\Users\admini\Desktop\OpenCPN 1715\plugins\s63_pi\s63_pi.dll
08:50:24:  Error: Failed to load shared library 'C:\Users\admini\Desktop\OpenCPN  1715\plugins\s63_pi\s63_pi.dll' (error 127: la procédure spécifiée est  introuvable.)
08:50:24:    PlugInManager: Cannot load library: C:\Users\admini\Desktop\OpenCPN 1715\plugins\s63_pi\s63_pi.dll

So I am unable to test my S63 charts

Yes, well, that is because dave is using a different compiler. All of the plugins have to be recompiled (or converted) The s63 is particularly difficult because it contains a binary program which I cannot recompile. As it stands this plugin also can not work on any arm processors. Only dave can provide binaries for it.

You will notice I have installers for various other plugins, and it's trivial to cross compile any plugin for any platform, although windows 64bit is broken.

* Ingested and displayed a 28 MB grib without a hitch

* Screen flickers when mousing over AIS target to display info, severe flickering when drawing routes and mousing over..

This is fixed now in git. (thanks dave and chuck)

* Dashboard display broken (part of the text is written and slowly erased) (no influence of OpenGl params, even without)

Ok, dashboard has issues. I think because I'm not using graphics contexts. shouldn't be hard to fix (without) or to look nice requires gdiplus dll which is not free software, so I was questioning if we could even use it at all. There is an open source version though I can try.

* I'm not too sure about that: Both procs seem used with intensive task such as displaying the current markers on a small scale chart, but proc usage doesn't go over 50%

* Testing in progress...

With accelerated panning it shouldn't use much cpu even with hundreds of current and tide markers.

[sailorF54]

Thxs for the dope... (you seem like a father waiting at the door of the maternity ward )

Although routes display is fine, dropping marks and displaying waypoints are NOT (black background)

[CarCode]

sailorF54,

* Selected at random (it's greek to me)
accelerated panning / texture compression / texture caching 64
for a low-end AMD Athlon Dual, 4 GB RAM, Radeon HD 3200 (with oGL 3.4, I think!)
Is that intelligent ?

Perhaps GLview might be of any help?
Available for Windows, OS X, iOS and Android from:
OpenGL Extensions Viewer - Download

GLview should show you your OpenGL GPU capabilities.

Gerhard

[sailorF54]

Originally Posted by CarCode

sailorF54,



Perhaps GLview might be of any help?
Available for Windows, OS X, iOS and Android from:
OpenGL Extensions Viewer - Download

GLview should show you your OpenGL GPU capabilities.

Gerhard

Thxs Gerhard
Yes, I saw your previous post about it, and have dowloaded and run it ('I think' meant 'If I remember correctly'. In fact it is 3.3)
But not being an expert, I don't know if my version of openGl is obsolete or not
The links to http://ati.amd.com/support/driver.html provided by this app to get new pilots does not seem to be functional... And I haven't had time to investigate further...

[seandepagnier]

I fixed the installer for all the languages, and the flickering bits and stuff. I can probably fix the dashboard problems you had. I'm mainly interested now in getting the google earth stuff working, because the mingw windows opencpn runs just as fast in wine on linux as the native linux opencpn. The visual studio version just crashes. This will allow linux users to use any plugins which only work on windows.

[bdbcat]

Sean...

1. Multicore CPU: OCPN was restricted to one core sometime in the distant past when there were problems with wxWidgets and inter-thread message passing. Probably no longer a factor. We did not care too much about this, since O only uses multi threads for the communication classes, and they are not CPU intensive. Lets turn the multi-cores back on, and see what breaks.

2. wxRegion. What a can of worms. We implemented OCPNRegion in response to serious performance problems with Mac Cocoa 64 bit native wxRegion code. I used an extraction of the gtk region structure and code, since it is FOSS. This code is only used on Mac at this point.

Since a region is nothing but a bunch of traversable rectangles, I see no reason not to use a generic OCPNRegion implementation for all platforms. Probably a good plan to derive from wxRegion, so that casting for interfacing with other methods will work smoothly, and we get automatic reference counting for fast (i.e.no-copy) passing as a function parameter.

I have seen similar unexplained memory corruption problems with Windows native wxRegion.

Thanks
Dave

[bdbcat]

Sean...

Mac OSX texture compression:

I poked around, semi-randomly, and found that the Intel OpenGL drivers on my Mac Mini need the following to work correctly with texture compression:

glChartCanvas:~505

        if( /*level >= base_level &&*/ !ramonly) {

Evidently, these OpenGL drivers need all the mipmap levels, starting from 0, in order to render properly. Renders white space if you only upload say levels 2 through 5, skipping the base(0) and level 1

Ever seen this before?

Downside is of course more tex memory usage. But not really too bad if one assumes that the user will likely zoom in to need level 0 mipmap eventually.

So, conditional for __WXOSX__, or implement for all platforms?
What say you?

Dave

[seandepagnier]

Originally Posted by bdbcat

Sean...

1. Multicore CPU: OCPN was restricted to one core sometime in the distant past when there were problems with wxWidgets and inter-thread message passing. Probably no longer a factor. We did not care too much about this, since O only uses multi threads for the communication classes, and they are not CPU intensive. Lets turn the multi-cores back on, and see what breaks.

Ok.

2. wxRegion. What a can of worms. We implemented OCPNRegion in response to serious performance problems with Mac Cocoa 64 bit native wxRegion code. I used an extraction of the gtk region structure and code, since it is FOSS. This code is only used on Mac at this point.

Since a region is nothing but a bunch of traversable rectangles, I see no reason not to use a generic OCPNRegion implementation for all platforms. Probably a good plan to derive from wxRegion, so that casting for interfacing with other methods will work smoothly, and we get automatic reference counting for fast (i.e.no-copy) passing as a function parameter.

I have seen similar unexplained memory corruption problems with Windows native wxRegion.

Thanks
Dave

Yes, this took me quite a lot of time being frustrated by the native windows implementation.

After looking at the actual implementation made me think it was also probably very slow. I do not derive from wxRegion anymore because before, some methods we didn't implement (like Offset) were calling the wxRegion version which had nothing to do with the gdk version. This is inviting future bugs if someone uses a method in wxRegion not in OCPNRegion. Deriving from wxObject gives the same reference counting benefits.

Originally Posted by bdbcat

Sean...

Mac OSX texture compression:

I poked around, semi-randomly, and found that the Intel OpenGL drivers on my Mac Mini need the following to work correctly with texture compression:

glChartCanvas:~505

        if( /*level >= base_level &&*/ !ramonly) {

Evidently, these OpenGL drivers need all the mipmap levels, starting from 0, in order to render properly. Renders white space if you only upload say levels 2 through 5, skipping the base(0) and level 1

Ever seen this before?

Downside is of course more tex memory usage. But not really too bad if one assumes that the user will likely zoom in to need level 0 mipmap eventually.

So, conditional for __WXOSX__, or implement for all platforms?
What say you?

Dave

Technically they are violating the opengl specification. I put this in, because when zoomed out, you have a great number of tiles, but don't need to use much memory to store the higher level mipmaps. Because of the compression caching, the mipmaps are pre-generated and already compressed, so they can be loaded very quickly. Zoomed in, you need the base levels, but, there are now fewer tiles, so the video memory use is the same.

For this reason, I find 2-6 megs of video memory is all that is needed for typical usage on a 1024x600 screen (larger screen uses the factor of the size of screen more) and caching off-screen tiles isn't much benefit anymore because loading from the cache is so fast. There is also no data stored in ram, so opencpn is no longer a memory hog and I could probably run 20 instances without noticing a slowdown.

If you store the base levels for all the tiles and zoom out far, it uses a lot of video memory. Without compression it can be well over 100 megs for a single chart in some cases. Are you saying this is a problem only with compression enabled?

For now, I would say use __WXOSX__, or even better, combine it with your particular driver if you find other osx systems without this problem, but if anyone else reports an issue, we must expand the conditional. I think it's yet another case of buggy graphics drivers.

For opengles, you must have all levels, from 0 all the way down to a texture that is 1x1 pixel, so that is implemented (and suboptimal) already.

The plan was to fix this, and provide even better performance by "manually" implementing mipmapping, by using a different texture for each level, then selecting the right one. Also you might notice I cannot free the base levels once allocated (this violates the opengl spec) without deleting the whole texture and rebuilding the mipmaps used. Also, many implementations likely allocate space for the base levels even if you aren't using them.

There was something almost like this before, but it also used regular mipmaps, and as a result required a lot more memory traffic as well as memory usage than it should have. Instead we must use a shader or multi-texturing to provide the linear_mipmap_linear blending (if available) or just using nearest mipmap on other platforms. Also uploading asyncronously using a background thread (possibly using pixel buffers), and a priority queue, so it would actually load the tiles many frames before they are needed in a background thread (while the user is zooming or panning) but this is complex, and I wanted to merge before diverging too far, so I don't plan on doing this soon.

[CarCode]

For those programming with OpenGL there are many documents available like:
https://developer.apple.com/library/...ngl_intro.html
and several TechNotes also like e.g.
"Technical Note TN2093_OpenGL Performance Optimization_ The Basics"
"Technical Note TN2178_Real world profiling with the OpenGL Profiler"
Furthermore there are tools like
OpenGL Driver Monitor.app
OpenGL Profiler.app

OpenGL ES is for embedded systems (ES) like iOS and not OS X.

Gerhard

[bcn]

Originally Posted by CarCode

OpenGL ES is for embedded systems (ES) like iOS and not OS X.

Gerhard

Gerhard,

what in your opinion does that imply in our context? Do you foresee problems, and in case, which?

Is the iOS/OS X distinction (embedded/General Purpose) you are stating only valid for the Apple World - which is by itself worth a consideration - or are you worried about the rest of the park as well?

Hubert

[CarCode]

Originally Posted by bcn

Gerhard,

what in your opinion does that imply in our context? Do you foresee problems, and in case, which?

Is the iOS/OS X distinction (embedded/General Purpose) you are stating only valid for the Apple World - which is by itself worth a consideration - or are you worried about the rest of the park as well?

Hubert

Hubert,
to get an answer you might read the code of the commits since 3.3.1712.
Gerhard

[bcn]

Gerhard,

if you are not happy with what you are seeing, state it and argue please.
I'm interested in a stable system, not in a discussion "who has the bigger brain".

And I'm not a coder.
There are arguments in this thread that are giving me the sensation that the code is getting machine dependent (exaggeration?). Something not very practical for maintenance and simplicity in my opinion. And not a good fundament for a rock solid software.

The zoo we have by combining multi architecture with a huge bunch of different underlying hardware is challenging.
How can we assure, that this big change or refactoring of the underlying basic functions will not jeopardize the stability of OpenCPN?
Is this stuff really ready for a Beta intended as a step for a stable release?

So, I'm worried, and would like to understand the actual state of the art better. "Read the code, stupid" is not helpful for me.

Hubert

[bdbcat]

bcn...

Not to worry too much about what is going on in the sausage factory. O is full of conditional code clauses to accommodate various hardware dependencies and driver bugs, and always has been. I wish it were not so, but this is the state of the art for multi-platform development.

The next Release version of O needs to be, and will be, a substantial improvement in functionality and performance. We just need to crawl through the issues, one-by-one if necessary, to get there.

Lots of the discussion here might not normally be seen in a public forum. It pretty specific, and requires a working knowledge of the code base. However, It is not secret info, and I think the more people know about the internals of O, the better.

Take heart
Dave

[Hakan]

Originally Posted by boat_alexandra

Can you verify that this problem didn't exist before? If so, could you use git-bisect to determine which commit broke things?......

Sean, delayed... sorry no usable wifi in the latest harbours.
No I can't verify, well I could by reinstall earlier versions or learn to use Git-bisect, but the Git pull from yesterday is better. I can still get the example back but it's only in one zoom level. And the collection of raster charts at the actual location is not easy. There are two specials covering the same area and using the same scale. The are a couple of detailed charts partly covering the specials area and at least two overviews also covering the same area. Two complete this CM93 is involved as well. So I would say OCPN is excused for the confusion. I suggest we leave this?
Håkan