Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Rating: Thread Rating: 3 votes, 5.00 average. Display Modes
Old 14-07-2011, 02:47   #226
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 4,016
Re: Charts II: NGA - 2700 Charts

One more to consider in deciding what to actually calibrate:
If we decide to do some more complex manipulation with the images as part of the preprocessing, like for example rotating the scans (and here I don't refer to 90 degrees where the math would be easy, but to a scan skewed something like 3.5 degrees for example), recalculating the REFs will add a fair amount complexity to our process. I would prefer to avoid it if possible.

Pavel
__________________

__________________
nohal is offline   Reply With Quote
Old 14-07-2011, 02:58   #227
bcn
Registered User

Join Date: May 2011
Location: underway whenever possible
Boat: Rangeboat 39
Posts: 2,759
Re: Charts II: NGA - 2700 Charts

Pavel,

working with the zoomed out details you will see that the border lines for calibration are actually covering 2 or 2.5 pixels. Therefore I'm with you in the doubt about the usefulness of calculations. At this moment I think that part has to be done manually.
In the case of the chart I was playing with, 1 px (at 50% reduction of resolution) is about 1m.
The lon/lat data on the charts, where are they actually referred to? Mid of the line, inner side of the line or outer?
Zoom in to 400% or even 800% to have fun...

Hubert
__________________

__________________
bcn is online now   Reply With Quote
Old 14-07-2011, 03:54   #228
Registered User
 
Netsurfer's Avatar

Join Date: Jan 2010
Location: Cologne, Germany
Boat: Beneteau Oceanis 331
Posts: 557
Re: Charts II: NGA - 2700 Charts

As it might not has become clear to all users how I think we should calibrate normal 1) chart images, here is an example:

Taken chart 37112 which is 19200 x 13400 px in size (these values are available for all charts from our DB).

Now I made 4 images out of the corners of the original image, each 1.500 x 1.500 px! This is fully automated by script! Processing time (without download the fullsize chart image) 10 seconds.

You could produce whatever image format you'd like to. The PNGs are max. compressed (9) but still slightly larger in file size than max. quality JPGs. But with file size around 400 KB this shouldn't really matter.

Now one who wants to calibrate this chart needs to download the 4 images (which could be packed in an archive). After downloading open them in a graphic program.

As on all "normal" charts we can get the lat/lng values for the SW and NE corners from the appropriate images. In our example these are:
  • SW: 4953'54'' N 050'24'' E
    converted to decimal degrees: 49.8983 0.84
  • NE: 5011'48'' N 131'00'' E
    converted to decimal degrees: 50.1966 1.5166
With these values we now have:
  • North: 50.1966
  • East: 1.5166
  • South: 49.8983
  • West: 0.84
So we can determine the lat/lng values for all 4 corners.

What's still missing so far are the according pixel coordinates.
Let's start with the image of the NW corner. With the help of our graphics program, we can determine that the upper-left corner is at (813|729). As this image is exactly at the same position as our original one the coordinates are also exactly the same.

Now let's proceed with the NE corner. Again with the help of our graphics program, we can determine that the upper-right corner is at (394|797). For the final pixel coordinates we have to do some simple math (could be also done by script later on). But just to clarify the system:

  1. Our original image is 19200 x 13400 px
  2. Our small image copy (NE) is 1500 x 1500 px
  3. Therefor (0|0) on our small image is = ((19200 - 1500) | 0) = (17700 | 0)
  4. Now (394|797) on the small image is (17700 + 394) | 797) = (18094 | 797)
So now we have also the pixel coordinates for the NE (upper-left) corner of our original fullsize image.

For the other two corners the procedure is identical.

But all we would need and which must be done manually is:
  1. The two given values on the chart for the SW and NE corner (no conversion needs to be done - simply use as given)
  2. The pixel coordinates of the 4 edges (also no conversion or arithmetic needs to be done when using the automatically produced 4 tiles)
  3. And of course the chart number which could be selected from a drop-down list or something like that
  4. Additionally we might gather some other information as well on this process, like projection and datum. At least for the charts where it's not Mercator and/ or WGS84
That's it. Entering these values allows for the rest to be done automatically by script.

Thanks

Gunther

1) "normal" charts - these are rectangular charts with no plan or insert where the lat/lng values for the SW and NE corners are given.
Attached Thumbnails
Click image for larger version

Name:	37112_9ne.png
Views:	60
Size:	365.5 KB
ID:	29616   Click image for larger version

Name:	37112_9nw.png
Views:	64
Size:	216.6 KB
ID:	29617  

Click image for larger version

Name:	37112_9se.png
Views:	57
Size:	278.4 KB
ID:	29618   Click image for larger version

Name:	37112_9sw.png
Views:	69
Size:	370.7 KB
ID:	29619  

__________________
Deutschsprachige Community- und Support-Website unter OpenCPN.de
Netsurfer is offline   Reply With Quote
Old 14-07-2011, 04:18   #229
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,247
Re: Charts II: NGA - 2700 Charts

Gunther
Sounds good, but
Quote:
The pixel coordinates of the 4 edges (also no conversion or arithmetic needs to be done when using the automatically produced 4 tiles)
will only work for a minority of charts produced, say this century.
In the general case we can't trust the pictures to be perfectly square, and we should not proceed building in this general assumption.

One example of pixelcount starting at the SW corner of 26345 Hamilton Hbr Bermuda. Don't bother about the actual values, just the relation between them

297,6462
270,347
8056,314
8084,6428

The accuracy of the end result is still OK
Click image for larger version

Name:	26345.png
Views:	57
Size:	56.5 KB
ID:	29620

Thomas
__________________
cagney is offline   Reply With Quote
Old 14-07-2011, 04:22   #230
Registered User
 
Netsurfer's Avatar

Join Date: Jan 2010
Location: Cologne, Germany
Boat: Beneteau Oceanis 331
Posts: 557
Re: Charts II: NGA - 2700 Charts

Thomas ...
Quote:
Originally Posted by cagney View Post
Sounds good, but

will only work for a minority of charts produced, say this century.
In the general case we can't trust the pictures to be perfectly square, and we should not proceed building in this general assumption.
No, you got me wrong. Of course we'll need the pixel coordinates of all 4 corners. What I meant by writing "also no conversion or arithmetic needs to be done when using the automatically produced 4 tiles" is that one doesn't need to do the arithmetic for the tile pixel coords => fullsize image coords.

Gunther
__________________
Deutschsprachige Community- und Support-Website unter OpenCPN.de
Netsurfer is offline   Reply With Quote
Old 14-07-2011, 04:31   #231
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 4,016
Re: Charts II: NGA - 2700 Charts

Very good work
Still it works just for the unskewed charts. But more than good for phase 1 aka. "Let's calibrate the easy ones first"

Notes:
  • Entering all the 4 corner coordinates has to be 100% mandatory - Reason: we can check the values and identify user errors and "suspicious" charts automatically
  • We have the total dimensions, so the user does not have to do the math at all and we can compute them server-side.

I will write a script producing the needed corner images.
Other info we should start collecting in this phase:
  • List of charts needing simple rotation by 90, 180, 270 degrees for the second iteration
  • List of charts that need special attention, eg. skewed scans etc.
__________________
nohal is offline   Reply With Quote
Old 14-07-2011, 04:40   #232
Registered User
 
LeaseOnLife's Avatar

Join Date: Apr 2008
Location: out cruising again
Boat: Sailboat
Posts: 1,046
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by Netsurfer View Post
As it might not has become clear to all users how I think we should calibrate normal 1) chart images, here is an example:
...
That's it.
With all this, we are trying to work around the fact that we can't easily work with full size images.

The calibrator still has to open the full chart to find values for
  • Chart Datum (MLLW, etc)
  • Lat where scale is correct for Mercator projection
  • ...
Then, in Region-6 I have lots of charts in the 1:300000 range where two charts are on one page, both rectangular. Plus all the charts with inserts. Maybe we need some statistics here, but there will be a large amount of charts needing a different workflow.

I don't think we get around reducing the images to a workable size. I am siding with Pavel and Hubert on this.
__________________
LeaseOnLife is offline   Reply With Quote
Old 14-07-2011, 04:41   #233
Registered User
 
Netsurfer's Avatar

Join Date: Jan 2010
Location: Cologne, Germany
Boat: Beneteau Oceanis 331
Posts: 557
Re: Charts II: NGA - 2700 Charts

Pavel
Quote:
Originally Posted by nohal View Post
I will write a script producing the needed corner images.
The problem hereby is how large the corner tiles should be because the border size differs from chart to chart. I only had a look at my local charts (the ones I am downloading) and just recognized that 1.000 px was to small. So I chose 1.500 but without any guarantee that this will be sufficient for all charts.

Quote:
Other info we should start collecting in this phase:
  • List of charts needing simple rotation by 90, 180, 270 degrees for the second iteration
  • List of charts that need special attention, eg. skewed scans etc.
To supplement the list
  • Info about the projection and datum
  • List of charts where the border PLYs differ from the REFs
And just for clarification: If a chart, like almost all of them, is just slightly skewed, this doesn't matter as long as we are going to gather the pixel coordinates of all 4 corners because only the pixel values differ not the Nort, East, South or West values!

The question here is what will produce the better result?
Just using the slightly skewed chart with the pixel coordinates and left the rotation to the presenting application (e.g. OpenCPN) or trying to rotate the chart before calibrating it?
My guess would be that the first one will be the better option - but just a guess.

Gunther
__________________
Deutschsprachige Community- und Support-Website unter OpenCPN.de
Netsurfer is offline   Reply With Quote
Old 14-07-2011, 04:53   #234
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,247
Re: Charts II: NGA - 2700 Charts

I'm not sure this has been mentioned before.
For low power systems, and other systems as well, the ruby script is superior as it saves the tiles temporary, while the python script just keeps everything in memory, ....and uses a lot of it.

The ruby script can also download a series of charts without user action by using a simple script. Here is the linux version
Code:
#!/bin/sh

for foo in   21584 22036 21601 21602 21603 21604 21605 22040 24480
do
./dezoom.rb $foo
done
Use your own chart numbers though!!!

Thomas
__________________
cagney is offline   Reply With Quote
Old 14-07-2011, 05:04   #235
Registered User
 
Netsurfer's Avatar

Join Date: Jan 2010
Location: Cologne, Germany
Boat: Beneteau Oceanis 331
Posts: 557
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by LeaseOnLife View Post
With all this, we are trying to work around the fact that we can't easily work with full size images.

The calibrator still has to open the full chart to find values for
No, not for the calibration itself.
Quote:
  • Chart Datum (MLLW, etc)
  • Lat where scale is correct for Mercator projection
  • ...
If you know any other way to get the info ...
I guess, or better hope, that everyone still has his downloaded charts. So if we have the info for these charts as a beginning ...!

Quote:
Then, in Region-6 I have lots of charts in the 1:300000 range where two charts are on one page, both rectangular. Plus all the charts with inserts. Maybe we need some statistics here, but there will be a large amount of charts needing a different workflow.
Such charts aren't what we call "normal" charts. I am already gathering infos about all charts with inserts, plans etc.!

Quote:
I don't think we get around reducing the images to a workable size. I am siding with Pavel and Hubert on this.
Sure we will reduce image size but not for calibration. As you like you can
  1. create local copies of your downloaded images reduced in size, quality etc. e.g. to look-up information about projection and datum
  2. wait until there are reduced size images available for download
IMHO the point is that we have to find a general working way for all users for every step of the process chain which needs to be processed. And in a way that the number of sources of possible errors is reduced to a minimum. Therefor it sounds reasonable to me to do one step after the other.

So calibrating images and reducing them in size are two different steps which are independent from eachother. But without the calibration data the rest is still useless. Therefor I think calibrating the files has a higher priority.

Gunther
__________________
Deutschsprachige Community- und Support-Website unter OpenCPN.de
Netsurfer is offline   Reply With Quote
Old 14-07-2011, 05:10   #236
Registered User
 
LeaseOnLife's Avatar

Join Date: Apr 2008
Location: out cruising again
Boat: Sailboat
Posts: 1,046
Re: Charts II: NGA - 2700 Charts

Gunter I think we talking past each other. Fully agreed to create the process. That could include an earlier resize where the output is helpful to the calibration process. Right now, I have almost no way opening full size images (=takes forever).
__________________
LeaseOnLife is offline   Reply With Quote
Old 14-07-2011, 05:21   #237
Registered User
 
Netsurfer's Avatar

Join Date: Jan 2010
Location: Cologne, Germany
Boat: Beneteau Oceanis 331
Posts: 557
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by LeaseOnLife View Post
Gunter I think we taking past each other. Fully agreed to create the process. That could include an earlier resize where the output is helpful to the calibration process.
Right now, I have almost no way opening full size images (=takes forever).
You do not need to work with full size images!
Let's say you are going to calibrate hundred charts for the beginning all you have to do is download about another 150 MB with the 4 corner tiles of each chart. Each tile is about 400KB.

By doing so we can assure that calibration is done on full size images (which will be best for accuracy) without the need to handle them but instead just 4 images each about 400KB.

Gunther
__________________
Deutschsprachige Community- und Support-Website unter OpenCPN.de
Netsurfer is offline   Reply With Quote
Old 14-07-2011, 05:57   #238
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 4,016
Re: Charts II: NGA - 2700 Charts

So, here's what my current automatic toolchain generates for a chart: Index of /ngacharts/nga/35000
  • 35000.jpg - original image
  • 35000.png - the product of preprocessing as described earlier and at https://github.com/ngacharts/ngacharts/wiki/The-Plan
  • the small pngs - the 4 1500x1500 corner areas - of course can be zipped so download in 1 step could be possible
Looks good for the simple cases so far. To have a look at the text to read projection info etc. we still can use the original NGA viewer, so probably not even downloading the preprocessed png version is necessary.

Pavel
__________________
nohal is offline   Reply With Quote
Old 14-07-2011, 06:16   #239
Registered User
 
LeaseOnLife's Avatar

Join Date: Apr 2008
Location: out cruising again
Boat: Sailboat
Posts: 1,046
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by nohal View Post
We still can use the original NGA viewer, so probably not even downloading the preprocessed png version is necessary.

Pavel
Pavel, great idea! At least for the folks working online.

Now, is there a firefox add-on to zoom into images and count pixels?
__________________
LeaseOnLife is offline   Reply With Quote
Old 14-07-2011, 07:16   #240
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 4,016
Re: Charts II: NGA - 2700 Charts

None that I know about that does what we need exactly. Gunther is our web technology guru, so maybe he comes with some brilliant idea. I'm out of this for years, but think that JavaScript got very powerful lately and looking at stuff like Photo editor online / free image editing direct in your browser - Pixlr.com what we need is probably doable, but have no idea how much work would it be to be able to capture the image click coordinates directly to our form.

Pavel
__________________

__________________
nohal is offline   Reply With Quote
Reply

Tags
charts

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off




Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 11:05.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.