Charts II: NGA - 2700 Charts

[bcn]

Originally Posted by paulr

I think quality is going to be a big issue and we need to either formalize QA somehow or else use a random 'many-eyes' approach. If everyone doing calibrations checked two others for every one they did, we would surely pick up on lots of errors.

As a case in point, chart 51100 is recorded as "no soundings datum", but the datum is there on the chart - LLW. This needs correction.

Paul

Thanks for the checks done. The 51100 is corrected.

What I noticed as well that one is getting more skilled in positioning with the time.
Anyhow, you are completely right that there should be a quality control/release system in place. A mutual check and a sign-off by more than one would be a possibility.

The other thing I'm missing right now is an indication at the charts about their status: there are some charts where the data are incomplete due to the "out of reach pixels". Or now the case of the 51100 that needs reprocessing.

And a reporting system for charts with problems as 51261 (chart incomplete, broken) or 51380 (calibration can't be edited under FF or Opera).

Hubert

[nohal]

Paul,
you are very right with the skew. It's an error to try to make the pixel coordinates be equal for the corresponding pairs of corners. Simply because the scanned charts are not perfect.
If you place the calibration marks to the real corners of the chart (The mark should be in the inner corner of the lines - trying to get it where degradation/anti-aliasing starts.), we compute the rotation needed to repair the picture as much as possible on the server. It's way faster and more precise than a human with Photoshop can be.
Do not try to "help" the system and really just place the mark to the corner of the chart, the rest is automatic.

The picture you posted in #538 is off by 2 pixels at full resolution of the JPG we downloaded from NGA - that's a size of 1 pixel on the KAPs we produce and difference in 1/100s of degree range for rotation. Before you get to need such a precision from the KAP, you probably switch to a larger scale chart or go out to make sure not to hit the rock.
Said that, I think we don't have to be overparanoid about exact placement of the calibration mark, as there are other errors we can't influence at all, that are larger.
But of course as everywhere else, the errors add up, so being too relaxed is not the best idea either.
To convert the error to real world, you have to count with the scale of the chart - it will be from hundreds of meters for the 1:10000000 charts in region misc to sub-meter for a 1:5000 harbor plan.

QA:
Right now the best we have is the "If you see an error, correct it" approach. I have some automatic tools to check the quality of the charts in the pipeline, but first need some more calibrated charts to tune the algorithms and a bit of time to implement them.
If you change anything on the chart, the KAP gets regenerated automatically. This is a community project, so we have to use it's strong aspects in having more eyes to do and review the work.
We will formalize the processes as time allows, actually anybody is welcome to do so - I'm for sure better writing code than documentation

All the problems reported (except 51380 - looking into it) should be fixed now (at least I hope so - getting a bit lost in the posts...)

Pavel

[bcn]

Pavel,

of course it would be interesting (and perhaps necessary) that the kaps are checked with different programs. I have only OpenCPN at hand, for example.

For regenerating charts: can we have maybe a time stamp at the download, to know if/when something has changed?

With respect to the 51261: the chart is still incomplete (in the NGA viewer ok).
Have a look on the thumbnail image!

I hope we don't throw too much crap at you!

Hubert

[nohal]

Originally Posted by bcn

of course it would be interesting (and perhaps necessary) that the kaps are checked with different programs. I have only OpenCPN at hand, for example.

That's for sure.

For regenerating charts: can we have maybe a time stamp at the download, to know if/when something has changed?

Yes we have this data in the DB, will try to extend the list with the chart status and timestamp of the KAP.

With respect to the 51261: the chart is still incomplete (in the NGA viewer ok).
Have a look on the thumbnail image!

Yep, I know, quite a few thumbnails are broken - they have not been generated from our images, but downloaded with the python script from the NGA site - that's the reason for some being broken.
The fullsize image is ok. The batch to replace the thumbnails with the scaled-down versions of our data is running, but as it involves almost 6000 resize operations, it will take a couple of hours to complete...

Pavel

[bcn]

Pavel,

great - no hurry.

What about the charts with the "pixel out of range" problem for calibration?
Any solution envisaged?

Hubert

[HappySeagull]

Originally Posted by bcn

Hi Happy Seagull,

perhaps we should organize who is taking care for which part of the charts to optimize our efforts?

If somebody could start with the 5xxxx series with 54000-54300 that would be fine.
I would look at the beginning after the charts 50000-53999.
Is that ok?

Hubert

Hubert

I'll leave 50000-5399 to you, then.

....I'm tempted to fill in "unknown" datum I know I can fix to WGS 84 but I am not, beyond noting them.
These could maybe be redone later and submitted to a subfolder of Opinion charts- a very useful option in areas charts are known to be flaky.We could end up with an excellent set of charts in the end!

as to corners,the mistakes Paul describes..perhaps that may have been one that was "out of bounds" and abandoned part way through?
I'm following the Nohal/Netsurfer example shown there as close as I can,then downloading "my" charts almost immediately to check them.

[paulr]

Different browsers display the calibration corners and red lines differently. Here are Chrome and Opera 11.5 for the same corner of 37025, with the same pixel settings.

Is one of these browsers unusable for this project (or both)?

[HappySeagull]

61036ut of bounds on east side..

[nohal]

paulr...
both are within the same pixel, both usable. The maximal difference it can possibly produce is 1 pixel at original resolution, which means less than 0.5 pixel in the resulting KAP.
We will try to have a look what causes the difference, but the priority is pretty low.

Pavel

[HappySeagull]

paulr..maybe make a chart with each browser,wait a few minutes,return to site,download em and compare in OpenCPN.

[nohal]

Originally Posted by HappySeagull

61036ut of bounds on east side..

Happy,
please set the chart status to "Rectangular, not doable in Phase 1 because the corners are not visible on cutouts" in these cases - I need this info for the moment when the solution to calibrate them will be ready - we will have to do some crunching on them and to save computing time, we need to be able to identify them.
If you do set the status properly, there's no more need to report the charts here.

Thanks

Pavel

[wdkester]

When I try to set the coordinates on 28006 it appears upside down. It is now rightside up in the NGA database whereas, a few weeks ago, it too was rotated 180.

[nohal]

David...
Thanks for pointing this out. We definitely need tools to check for updates on the NGA site.
Looking at this particular case, they just replaced the files in Index of /NGAViewer/28006 on 30th August without reflecting it anywhere, so we would have to periodically parse the html code of all the almost 2800 directory listings and check the date next to ImageProperties.xml to catch cases like this.
Not too complicated to write, but someone has to do it. Any ruby/python/php/c/whatever that runs on Linux coders here?

Pavel

[HappySeagull]

..sorry,Nohal.but no worries, I've been doing that in the Chart Cloud .dunno why I mentioned that one.a frustration leak after getting all done but 2 corners

[bcn]

Happy,

you will find some more charts which do not allow to access all the corners for calibration.
In general if there is a too broad border on the chart this happens - right side, top, bottom..
Lat/LOn you can get always with the NGA-viewer, but for the rest we will have to wait for a different solution.

Hubert