OpenCPN Version 1.3.5 Beta Technical

[cagney]

Dave

One possibility with 6951 is to add many more REF/ points.
In libbsb there is bsb2tif, that creates a tiff. This can be used to do a proper geo-referencing, then a new kap can be created.
This 'round trip' works, I have tested using the existing header.
Not elegant, lots of work to do all those affected SHOM charts, but possible. But of course, this still leaves the charts skewed.

Thomas

[dacust]

Are the SHOM charts available for download somewhere? My search only found places they have them for sale.

[Psyches]

Originally Posted by bdbcat

Maybe you could create some diffs against the current CVS, and I'll try to merge them.

If this makes sense, please email the diffs to me directly. email address in help->Authors tab.

Excellent! I'll gather things up and send them ASAP. I'll briefly list what I did & why here - a few changes had ripple-effect, mostly for your #2 below; hopefully this list will make the diffs easier to follow. Also, you may want (or want me) to change some names. I tried to keep changes inside OpenCPN to a minimum, but e.g. ChartBase was already defined and used many places, yet has the main chart UI interface...so I created "Chart" that ChartBase inherits from. Maybe should be ChartBase and ChartBaseUI or something?

Also I didn't finish the full leaf-class chart separation (the core of your #2 below)...I was planning for next week. So I still can't create a chart without it needing lots of UI stuff. I was going to use multiple inheritance to facilitate, but some other approach with delegation and a separate class reference should be just as good - I think this is what you meant but I'm not sure.

Changes:

(a) move toward letting another app use the chart database:

- moved core chart enums to chart.h, and pending reorder of ChartTypeEnum (so UNKNOWN == 0, DONTCARE == 1, DUMMY == 2 for the future); I planned to address this db format change automatically due to changes below
- moved key non-UI ChartBase methods to Chart
- separate ChartDB into two classes: ChartDatabase class is the new one; ChartDatabase doesn't have a notion of stack or currency, it only reads/writes/updates/creates the database and acts as a Chart factory. ChartDB keeps stack and currency, and uses ChartDatabase to get charts. I think better names could be found for the division, but I didn't want to make too many changes.

(b) purify, simplify, and ensure I wasn't screwing something up

- moved Chart & ChartBase toward container classes rather than interfaces (removed virtuals, added more fields) since I found most virtual overrides just used fields in the parent. The biggest impact was to cm93comp, see below
- moved all public and protected fields into private and added accessor methods, so I could tell & enforce who used what, when...and change the algorithm transparently if needed. The accessors are mostly defined in the header and can be inlined (aren't yet) but I didn't find one that really needed inlining.
- in Chart, made methods, parameters, and results const wherever I could, to assert no surprise changes were happening (e.g. COVR pointers)
- converted pointers and by-value params to references in some fields and formal params, to assert validity and remove visible indirection
- moved some field setters to protected, to assert that only the hierarchy needs to modify the values
- removed the color scheme on the virtual Init method in Chart, since it is UI-related. I added a non-virtual Init into ChartBase that takes ColorScheme, and in OpenCPN, it calls that one; that one calls the virtual and does color scheme init. I found that code in a few places, and just consolidated into the one place.
- converted (carefully) ChartTableHeader and ChartTableEntry to objects. They now handle their own serialization/deserialization/cleanup, to help consolidate. I started moving ChartTableEntry to accessor-method use, so e.g. pFullpath became a method that first adds its directory to its delta path, and during database read, it replaces the common part of the path with the shared directory prefix info.
- rather than an array of structs, changed the ChartTable as a wx array.
- keep the directory list in ChartDatabase, so ChartTableEntries can refer to it
- abstract the access to the directories, so they can keep stringified checksums (separated by ":" as in the original code)
- added a SetCurrent/GetCurrent to cm93comp; it calls the needed Chart methods to change the current "virtual" chart. I moved the current chart field in this class to private and renamed so I could find all places that changed the current chart. Then I changed those places to call SetCurrent().

1. I have been considering changing the binary chart database file format into an XML human readable/editable format. Of course, it will get larger on disk....
Internal representation may grow, too. I need to add some more fields to database entries.

What is the issue here with Mobile?

for XML as the main chart database, I see pros and cons. The biggest pros: it's readable and human editable, parsing is readily available. The cons: it may be huge for a large chart set. Size may not be the biggest concern; I expect reading the entire thing upon every OpenCPN load could visibly slow startup. My chart db is 1.2MB as binary; from prior calcs I'd say ~260K is string, the rest is number, or ~1MB. I'd guess 10x for XML, or ~13MB read and parse on every load. Parsing will be really fast, but reading 13MB and storing as internal structure will still take a bit of time. It'd be interesting to prototype to see. What do you hope to gain?

Depending on what you want, there are some intermediate points. Reading binary will always be a lot faster. Rather than serializing per-struct, you could serialize per-field (hand serialization) and have more flexibility in version updating. I think keeping read/write in the "object" (ChartTableEntry) helps isolate and makes it robust. But you could also add ReadXML and WriteXML methods...effectively supporting a full chart db import/export approach while keeping binary as the native mode. Again, depending on what you want.

For Windows Mobile, I don't think XML would be any more issue than for other platforms; of course you won't want a DOM-style parser (I think wx is pull-parser style? Haven't looked enough yet). WM has somewhat limited process memory, and OpenCPN keeps the whole chart table in memory. Changing that would be pretty complicated, I think, and I'm not convinced it's necessary. The chart stack isn't a concern because it should always be very limited in size. The cache will need to change for WM, but no big deal; if I remember, we assume MSW has unlimited VM, and WM doesn't. But keeping lots of chart entries all with duplicated strings can take a lot of space. So optimizing the in-memory chart entry helps Windows Mobile. Hope this makes sense, and I haven't misunderstood anything.

2. Also been meaning to "purify" the chart classes, for just the reasons you enumerate. Probably have to define some kind of global_UI_config class, and pass a reference of this to the chart class by method or ctor, so that global statics are not needed, and it will do the "right thing" without a UI... More thought here...Now I have some motivation
Lets see your code...

Makes sense...I'm interested in more of your thoughts there. Along this line, something else that would make sense for UI-less work, is to replace the direct progress dialog invocations with a method callback or pure virtual class (interface); that would allow easy separation of UI for the key low-level chart database and s-57 operations.

Mark

[cagney]

'robert' the signature that raised the questions about the SHOM chart accuracy compared how different charting program showed he latitude around Cap d'Armes lighthouse. The official lat is N 42 49.02.
He compared charts in 1:250,000 , 1:500,000 and 1:2,250,000.
He found that OpenCPN was 13 M out in the small scale chart, 6757.kap. This chart also only has 4 REF points. The header gives PP=0 while the text on the chart gives the scale at N 41.5.
When checking with the latest beta I found the error in latitude to be 0.4'. This makes me think that 'robert' used 1.3.4, and that OpenCPN:s new algorithm kicked in here.
Comparing the 1:250,000 chart 6951.kap I found an error of 0.012' of latitude (good with over zooming in this case ).
OpenCPNs values on these two charts are comparable to 'roberts' published values for SeaClear, slightly better on the 1:250,000 chart and 0,2' worse on the smaller scale chart.

Thomas

[bdbcat]

Psyches....

Mark:
All sounds good. It will be nice to get some help to unwrap the spaghetti code that results from an evolving codebase with a moving target.

Lets move the techno discussion off to private email. Sometimes the sausage factory is not a nice place to visit. Shouldn't take too long to get this integrated and into the CVS base for those interested.

Cagney, Marco, Sinbad7, and other chart producers: the main upshot of this work for you is the chart database "update-in-place" effort. This will mean that when you create or re-create a KAP chart and copy it into an already registered directory, opencpn will detect the change and load or reload the chart parameters for immediate use, without having to reload any other charts in that directory, or any others that have not changed. Saves lots of time, especially if you have ENC_ROOT in your tree.
Coming soon....

Dave

[Ptizef]

[IMG]file:///C:/Users/JEANPI%7E1/AppData/Local/Temp/moz-screenshot.png[/IMG]
Dave
I can confirm that grib files are not sorted by modification date : see below my directory and how its are sorted by OpenCpn

[bdbcat]

Ptizef....

Just to confirm your Beta version, please?

Thanks
Dave

[Ptizef]

1.3.5 build 1122
Jean Pierre

[bdbcat]

Ptizef:

Jean Pierre....

Thanks for the info. I found the bug in sorting GRIB files, fixed for next Beta. Funny that it only is evident on Windows. I think it depends on the file system type.

Anyway, the fix is clear enough.

Thanks again
Dave

[bdbcat]

Psyches...

Mark...

Any time frame on the diffs you are working on? I hesitate to do too much more here without a merge of your stuff....

Thanks
Dave

[Psyches]

Originally Posted by bdbcat

Any time frame on the diffs you are working on? I hesitate to do too much more here without a merge of your stuff....

Dave, sorry for the delay; was trying to finish and family time took the place of work the last few days. I'll get them to you tomorrow (Mon 11/30), ok?

[Sea Otter]

Hi guys,
I am a new fan of OpenCPN and have the beta version 1.3.5 1118, will there be any announcement when a new version comes out or is ready to be tested?

I must admit that this easy program still contains a lot of what the expensive commercial nav programs have. I have been using MaxSea until now (still using it on the old laptop with XP installed) but on the new machine with Windows 7 I couldn't get the GPS and AIS working, although Win 7 found the correct com ports. So, I just tried the OpenCPN for fun at first, but as it worked straight away finding GPS and AIS I continued exploring and now I am just waiting for all the improvements that are coming. The Grib reader was a hit, nice work bdcat!!!!
I have found one little thing that does not work on my machine and that is I cannot scroll down the list of waypoints in the route properties window. I can scroll sideways but not vertical. Anybody else found this?

[Sea Otter]

Another question, not exactly related to OpenCPN. The famous CM93 worldwide maps I got from a friend are on 4 CD's. When I copy them on to the HDD root C:\ There are 9 regions, But it seems like there are double copies of some files from region to another region. The computer asks if I want to skip the copy or to install it beside the other one.

How do owners of these maps install them on the HDD?

[Viking Sailor]

Overwrite the smaller version of a chart with the larger version of the same chart. This is automatic if the charts are extracted in the right order with overwrite files selected. I extracted all of the charts into one directory beginning with zone 1 part 1 and ending with zone 8 part 4.

Paul

[utinam]

HI, just arrived in your community and interested in all open source..
Opencpn seems very intersting and I downloaded your last version working with Cmap93.
I am with windows XP and the depths do not appear.
Can you help
Thks