Charts II: NGA - 2700 Charts

[wdkester]

25608 is marked 90 deg skew but hasn't yet been rotated. It's composed of rect charts so can wait until we have a solution for those.

[Tia Bu]

The conversation above about how to "release" the charts illustrates, to me, why this project would work better as a chart "wiki".

Nothing can make these official copies of NGA charts. Yet many, many of these charts have already had exhaustive scrutiny and quality checking by several sets of eyes during this process. I suspect that many of them have gotten more consideration here than they have at NGA over the past couple of decades.

Why not, at some point a bit farther into the future, set this up as a chart wiki with a clear introductory warning, explaining how the charts came to be here, warning that they should be carefully checked by users and not relied upon as a sole aid to navigation, and disclaiming any responsibility for errors?

We could continue to allow corrections to the .KAPs using the processes we already have in place. Those processes clearly seem to yield more accurate .KAPs over time. If it is feasible in a web-based system, we also might add the capability to position a very limited number of symbols (in a color or shape that would readily be recognized as an add-on to the NGA chart). Those could be symbols for soundings, aids-to-navigation, obstructions, or notes. An early use of those symbols could be to update Notice to Mariners information on the charts.

Eventually, those postings could be used by cruisers in the field to add soundings, etc., to the charts.

Although there has been a LOT of study and debate on the subject, it appears that Wikipedia is about as accurate as the Encyclopedia Brittanica. I suspect the real news here is that the Encyclopedia Brittanica is not the monolithic source of truth we all imagined it to be. After working with the NGA charts up close and personal for a couple of months, I realize I may be learning the same about U.S. Government charts.

A wiki really might be a good way to get this project into the hands of the boating public. An easily accessible source of free, reasonably accurate charts that can be used with caution and prudence is arguably better than the current situation, i.e., very expensive individual charts that, after all, are reasonably accurate and should be used with caution and prudence.

[HappySeagull]

a good and useful idea,TiaBu and true too...but all that happens to these after,and wherever it happens,there need be originals. and these " originals" could use watermarks AND md5 sums too....

.I bet there's a ton of mp3's out there all sporting the same filename but wildly different to listen to ...I'd bet that almost everyone is looking for the original first,however improved the copies are.

[dacust]

Originally Posted by HappySeagull

a good and useful idea,TiaBu and true too...but all that happens to these after,and wherever it happens,there need be originals. and these " originals" could use watermarks AND md5 sums too....

.I bet there's a ton of mp3's out there all sporting the same filename but wildly different to listen to ...I'd bet that almost everyone is looking for the original first,however improved the copies are.

Absolutely!

I wouldn't mind having access to wiki updated charts. Love it, actually.

BUT, I'd want the originals so I could compare and see what was added/removed/changed.

If I had to choose and could ONLY have either the originals OR the updated ones? I'm one of the old curmudgeons that would choose the originals. but I'd rather have both.

Hopefully at some point we'll get the user definable chart groups requested in Flyspray. Then I could have a group for NGA - Originals, and another for NGA - Wiki updated. Then with a single click, I could jump between them on the screen to see the differences.

-dan

[recri]

NGA 2700 chart summary for Thu Oct 13 01:33:20 GMT 2011
processed: 2343 charts
evaluated: 2273 charts
[ omitting everything else, all below 5 parts per thousand ]
ratio: mean 1.00437 stddev 0.01400
>25*sd: (0.35006) 51082 28042
>13*sd: (0.18203) 54369
>3*sd: (0.04201) 62080
notdone: skipped 1 charts: 22545
notmercator: skipped 69 charts

chart 22545 is a Galapagos Island which needs corner coordinates interpolated.

Personally, I've been wishing for a pair of onscreen dividers, but all I've found are rulers.

[Tia Bu]

Originally Posted by dacust

Absolutely!

I wouldn't mind having access to wiki updated charts. Love it, actually.

BUT, I'd want the originals so I could compare and see what was added/removed/changed.

If I had to choose and could ONLY have either the originals OR the updated ones? I'm one of the old curmudgeons that would choose the originals. but I'd rather have both.

Hopefully at some point we'll get the user definable chart groups requested in Flyspray. Then I could have a group for NGA - Originals, and another for NGA - Wiki updated. Then with a single click, I could jump between them on the screen to see the differences.

-dan

Aye, there's the rub! We'll never have "the originals". We'll only have rather strangely developed copies. And we don't have any "authority" to say when we have the copies "finished". The one thing we do have- already- is a process that yields continuous improvements in quality through the anonymous, self-directed, and largely unregulated efforts of volunteers.

Basically, our quality control at this point seems to be addressing three issues (I think): 1) Is the graphics file we constructed geometrically correct? 2) Did we calibrate that file correctly as we converted it to a KAP by a) correctly inputting the pixel coordinates of the corners and b) correctly entering the lat/lon coordinates? and 3) did we correctly enter the metadata?

But then other issues have come up in this discussion. We have some charts where we have debated the right correction to adjust the chart to WGS84 datum. And we have come up with at least one chart that we have determined, by using Google Earth, cannot be corrected to WGS84 datum at all.

I suspect these issues of quality control will not fully be resolved until we have worked with these charts for months to come and people have used them in the field and identified problems.

Could we even get to a point where we could watermark these things and declare them to be "originals"?

[dacust]

Originally Posted by Tia Bu

Aye, there's the rub! We'll never have "the originals". We'll only have rather strangely developed copies.

When I say originals I mean before allowing people to add new markings, buoys, depth soundings, etc.

[recri]

More than a wiki, we need something like git.

We need chart version management, because we want to see how the chart became what the chart plotter is displaying.

And we need the distributed synchronization that git provides. So we can pull updates into our local copies of the charts and push any corrections we've made to our local copies back to the shared repository for merging.

And we also need branches, so that we can manage inconsistencies where there are two or more ways that a fuzzy chart might be resolved, but we can't say which is right, so we store the reasonable possibilities rather than one best guess.

[HappySeagull]

recri,ie 22545 :
I had to download the jpeg and "Map-Cal" it to get some corners to work.Turns out it's a teeny bit skewed clockwise,so lower-left corner published latitude really didn't work that well...
It's looks ok now in O, if not perfect....

[HappySeagull]

I agree,somewhat.
My post above shows the charts are still "wikis" now, albeit inside parameters.
When the charts are "Locked",they become "original".

So,when will they be locked?There's plenty of charts like 22545 above that can be improved in this way...a slight twist of the image,a closer fudging of corners to get a grid.That's really what we're doing now.A "best" grid on a "best" image.
Shifting the grid to agree with WGS 84 is a slightly different endeavour but since we are deferring to the on-chart corrections already,the grid is defined as "WGS84 where possible"
Donated wgs grid corrections are merely "reliable sources".They are outside the "original" but I think the branch is very near and they might even produce locked versions.

Another nearby branch incorporates the available NTMS.

A whole other trunk is adding/deleting rocks,docks,buoys,islands ...here's where it gets messy but maybe the best info will appear.

[Tia Bu]

Originally Posted by recri

More than a wiki, we need something like git.

We need chart version management, because we want to see how the chart became what the chart plotter is displaying.

And we need the distributed synchronization that git provides. So we can pull updates into our local copies of the charts and push any corrections we've made to our local copies back to the shared repository for merging.

And we also need branches, so that we can manage inconsistencies where there are two or more ways that a fuzzy chart might be resolved, but we can't say which is right, so we store the reasonable possibilities rather than one best guess.

From what I was able to read quickly about GIT, your idea sounds good. It sounds like it might help handle the issues we're bouncing around above. Would it be possible to design a user interface that the average Joe could use to download some charts, without having to revisit all the nitpicking other people have been doing on these files?

[z.stadler]

Originally Posted by HappySeagull

Shifting the grid to agree with WGS 84 is a slightly different endeavour but since we are deferring to the on-chart corrections already,the grid is defined as "WGS84 where possible"

You probably don't mean "Shifting the grid to agree with WGS 84" literally, right?
The grid of an unknown datum is located at different physical location than the WGS84 grid.

[HappySeagull]

I mean what we've been doing.If the chart offers a correction to agree with WGS Satellite navigation ,we do it so that it's in the chart header as a DTM...the printed-on-the-chart grid is an artifact related to other artifacts(pencils,parallel rules-a whole other medium)We are adapting these to a new use.
Hopefully,nobody is including any corrections away from WGS!
Several of these Unknown datums can be discovered.I've discovered two,because I have the original charts-in paper if you like!and the datum is not "Unknown" on them.But I only noted it.Hopefully,it will be done somewhere,because these versions are out quite a bit,given OpenCPN opens em as WGS pretty sure she does!
The "grid"?well,openCPN can show this and many others too but it is there visible or not.

I haven't tested these in other apps yet to see what "unknown" means to them!OpenCPN is by far the newest I have,so others will have to swing the lead on this.

[cagney]

What we have achieved, so far. A different view.
Some Arctic and Antarctic charts are not visible.





Thomas

[bcn]

Originally Posted by cagney

What we have achieved, so far. A different view.
Some Arctic and Antarctic charts are not visible.


Thomas

Yes, looks great (and making us proud perhaps)!

Prepared to release (what)?

My sensation right now is that the collection is quite good and that we are on a correct track - and that we can deliver a lot more.

Wiki is not the way to go for me - we are not talking about "opinions" (sorry) but about charts that are operative/ready for use.
A correct solution is more about for instance a Debian/Linux approach.

Releasing, being sure what you are offering to use - a stable source.

So, I would like to come back to the question: how to release and how to organize this process.

Making add-ons, release cycles, branches perhaps etc. is a process apart - an interesting one - a second/third step. Or a thread in parallel if we have the capacity to run it.

Hubert