Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Rating: Thread Rating: 3 votes, 5.00 average. Display Modes
Old 13-07-2011, 09:25   #211
Registered User
 
Netsurfer's Avatar

Join Date: Jan 2010
Location: Cologne, Germany
Boat: Beneteau Oceanis 331
Posts: 557
Re: Charts II: NGA - 2700 Charts

Thomas
Quote:
Originally Posted by cagney View Post
REF and PLY
We should get the exact corner lat&long, from the printed information in the SW and NE corners of the charts.
Yes, and from this we can simply calculate the NW and SE points, at least for the charts where the two corner coordinates are given (not for all charts).
Quote:
We also need the corresponding pixels to a very good accuracy. I zoom in at 200% to sort the pixels out.
For most charts the 4 corner will do, but not for all, there are a few crap shots around!! Some charts - mainly inserts, needs more PLY points as the area is not rectangular.
Quote:
Using the 4 corners has an other advantage. QA at a glance, the PLY line should follow the inner border very accurately.
For rectangular charts this is very simple. I propose to calculate the PLYs by setting them a little bit smaller (about 5-10 pixel) than the outer edges of the chart. The lat/lng values could be simply calculated.
Quote:
The quality should be tested further by zooming in and having the mouse pointer over a printed lat/long crossing, and read the values. How accurate is acceptable? I suggest that up to and including the 4:th decimal should be correct with normal rounding off rules. Repeat for ~4 similar points spread out. We mustn't compromise on this, and produce a chart as accurate as possible.
When the SW and NE coordinates are given why do you want to round the decimal degree value?
Quote:
Another thing is that if we make the picture a different size, only the pixel count for the corners change. There will never be any doubt as to exactly which reference points that were used originally.
I also suggest that we follow a "semi-standard" and start REF and PLY in the SW corner and proceed clockwise.
Hehe ..., I suggested several times before to use them counter-clockwise.
But anyway, whether clockwise or counter-clockwise doesn't matter in the end as long as point follows the other for the rectangle/ polygon.
Quote:
Quite a few charts have inserts, named xxxxxxA etc. For these, the corner positions, often more than 4 and the corresponding pixel count must be retrieved, as well as the Name and scale for the inserts.
The name and scales are available in the pdf catalogs for each region, so hopefully this can be automated as well.
I don't think so ...!
Though you might convert the pdf to another format which you are able to extract the information from as text it would be still difficult to set them up to the correct chart.
In generall: How to deal with such charts?
In my opinion we
  • use the original image for the "main" chart with the PLYs set correctly
  • have to copy rectangular images from the original one for each plan
    these ones could be named something like 37235_plan_[a-z].jpg
  • calibrate and set PLYs for them
In my understanding there is no other way as to make a single chart (kap) out of each plan integrated in a chart.
Quote:
We also have to make up our minds how to handle those chart pictures that are turned 90 degrees. In the end its just a question of the pixels, nothing else changes.
Why not simply turning them before any other processing? This is possible without any loss of quality to the original JPG.

For the charts where the SW and NE corners are not given:
The attached image shows the upper left corner of a chart. As you can see in such cases it won't make much sense to use the edge as you have to guess the right lat/lng values.
And about the accuracy: The more you zoom in the more pixel are used for a line. So it's the small pink '+' sign in the middle which you should use for the pixel coordinates.

Gunther
Attached Thumbnails
Click image for larger version

Name:	nga-charts-corner-01.png
Views:	59
Size:	21.5 KB
ID:	29597  
__________________

__________________
Deutschsprachige Community- und Support-Website unter OpenCPN.de
Netsurfer is offline   Reply With Quote
Old 13-07-2011, 10:12   #212
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,125
Re: Charts II: NGA - 2700 Charts

Gunther
See comments below.
Quote:
Originally Posted by Netsurfer View Post
Thomas

Yes, and from this we can simply calculate the NW and SE points, at least for the charts where the two corner coordinates are given (not for all charts).



For rectangular charts this is very simple. I propose to calculate the PLYs by setting them a little bit smaller (about 5-10 pixel) than the outer edges of the chart. The lat/lng values could be simply calculated.

Quote:
Why?? Why not just follow the inner edge?
When the SW and NE coordinates are given why do you want to round the decimal degree value?

Quote:
I'm talking about checking the geo-referencing. How OpenCPN interprets the available information. There are no guarantees that the chart hasn't been warped or what not...during the scanning for example. Neither is there a guarantee that the chart has been produced as a 100% accurate Mercator chart.
Hehe ..., I suggested several times before to use them counter-clockwise.
But anyway, whether clockwise or counter-clockwise doesn't matter in the end as long as point follows the other for the rectangle/ polygon.

Quote:
Sorry! Maptech started.
I don't think so ...!
Though you might convert the pdf to another format which you are able to extract the information from as text it would be still difficult to set them up to the correct chart.

Quote:
I'll have a look at this.
In generall: How to deal with such charts?
In my opinion we
  • use the original image for the "main" chart with the PLYs set correctly
  • have to copy rectangular images from the original one for each plan
    these ones could be named something like 37235_plan_[a-z].jpg
  • calibrate and set PLYs for them
In my understanding there is no other way as to make a single chart (kap) out of each plan integrated in a chart.

Quote:
I suggest that we simply name the plans 37235_A.jpg, The plans are generally named A, B etc on the charts.
Once a plan is copied, we just treat it as another chart.
The space where the plan was/is on the original should be whited out or be filled with the surrounding "land-color"
Why not simply turning them before any other processing? This is possible without any loss of quality to the original JPG.

For the charts where the SW and NE corners are not given:
The attached image shows the upper left corner of a chart. As you can see in such cases it won't make much sense to use the edge as you have to guess the right lat/lng values.
Quote:
True. But this can be handled. More manual work though. In some cases we may need to have a comment on exactly which reference points that where used. Generally a procedure could be to use the SW+NE corners of the largest available rectangle of printed grid-lines on the chart-plan, for geo-referencing, then load in OpenCPN and get the lat&long for the PLY polygon. Yes a PITA. In some cases even worse.
Hopefully something simpler can be worked out.
MapCal could help here when doing it manually, but not for automating the process.
And about the accuracy: The more you zoom in the more pixel are used for a line. So it's the small pink '+' sign in the middle which you should use for the pixel coordinates.

Gunther
Thomas
__________________

__________________
cagney is offline   Reply With Quote
Old 13-07-2011, 10:20   #213
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 3,903
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by Netsurfer View Post
About the colors: I have already seen charts where the land area is yellow (e.g. chart 700). And on many other charts the land area is grey. Or is this eliminated by your noise reduction scripts?
Not at all and I don't think it should be doing anything about it (although it of course could be handled once done manually for the first time)
Quote:
I also do not understand why you are creating a 32 color image?
When converting to a palette image format there will be a maximum of 256 colors. Also it is possible to get the number of used colors. Wouldn't it be more efficient to get this value first and afterwards producing an image with the "optimal" color count (assumed it does not exceed our 127/128 max)?
It's an experimental value resulting in a pretty good combination of quality and size. The charts are black, white, blue and gray in most cases, it means the palette ideally would be even smaller (even if you count the antialiasing half-tones) But as it comes from a scan, it's non-trivial to achieve. No dogma here, our experimenting has to come up with the ideal settings.
Quote:
As the charts are scanned from paper charts some of them are also slightly skewed/crooked.
Of course some charts need special "treatment" Not much can be done automatically, but the workflow I'm working on takes into account that for some charts there will have to be special scripts handling them. That's why not everything can be blindly generated from the database. An that's why we will need to store stuff in git as well.
Quote:
What does 'QA' means (quality assurance)?
Hmmm ..., as much as I like the idea of automation I couldn't think of any other way as doing the calibration manually/ by hand. But maybe you have an idea ...
Yep, for that we need people's eyes and brains. Same like for identifiing the charts demanding special treatment.
Quote:
Addendum: And there are also charts with harbour plans or several parts of a river etc. (good examples are 37235 and 37246 / upload coming soon)!
Same as above. But same as above, once we know how the chart is composed, we can script a machine to do it next time we need it.

Pavel
__________________
nohal is offline   Reply With Quote
Old 13-07-2011, 13:28   #214
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,125
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by cagney View Post
Thanks Jim.
I'm testing the version below right now.
Attachment 29596
Thomas
No that wasn't right!!
Use this version instead, has run 3 in a row by now.
dezoomify2.py.doc

Thomas
__________________
cagney is offline   Reply With Quote
Old 13-07-2011, 23:22   #215
Registered User
 
LeaseOnLife's Avatar

Join Date: Apr 2008
Location: Around once, back to work, for a while :-(
Boat: Sailboat
Posts: 967
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by Netsurfer View Post
For rectangular charts this is very simple. I propose to calculate the PLYs by setting them a little bit smaller (about 5-10 pixel) than the outer edges of the chart. The lat/lng values could be simply calculated.
I don't think so, maybe I misunderstand. During my calibration I found the charts are not scanned perfectly, lines are not perfect horizontal or vertical. Don't calculate REF points, only humans can define a REF point. And YES, if possible we should use the corners.

If the chart is rectangular, and all 4 corners have been used for REF, then surely we can re-use them for PLY.

About inset charts:
Quote:
Originally Posted by Netsurfer View Post
In generall: How to deal with such charts?
In my opinion we
  • use the original image for the "main" chart with the PLYs set correctly
  • have to copy rectangular images from the original one for each plan
    these ones could be named something like 37235_plan_[a-z].jpg
  • calibrate and set PLYs for them
In my understanding there is no other way as to make a single chart (kap) out of each plan integrated in a chart.
Makes sense to keep the complete size chart for each inset chart, otherwise the splitting and cropping will be done differently every time and REF/PLY can't be re-used. To make it repeatable, keep the full rectangular chart, white-out the areas not used, as suggested by Thomas. Image compression should take care of the white space and the file should not be that much bigger than a cropped chart.

One more thought: Should we perform and save the calibration (REF & PLY) for the full size chart, before resizing? When resizing, the script could recalculate the pixels from there? That would allow us to resize to any smaller size without loosing accuracy. Caveat: MapCal on my netbook scrambles to work on xx-MByte charts.
__________________
LeaseOnLife is offline   Reply With Quote
Old 14-07-2011, 00:25   #216
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 3,903
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by LeaseOnLife View Post
One more thought: Should we perform and save the calibration (REF & PLY) for the full size chart, before resizing? When resizing, the script could recalculate the pixels from there? That would allow us to resize to any smaller size without loosing accuracy. Caveat: MapCal on my netbook scrambles to work on xx-MByte charts.
VERY good question. Of course it would be the best and most accurate, but I really fear that for practical reasons we will have to calibrate the preprocessed 50% charts (if 50% is the size we decide is the one we will use for our charts).

Pavel
__________________
nohal is offline   Reply With Quote
Old 14-07-2011, 00:36   #217
Registered User
 
Netsurfer's Avatar

Join Date: Jan 2010
Location: Cologne, Germany
Boat: Beneteau Oceanis 331
Posts: 557
Re: Charts II: NGA - 2700 Charts

Pavel ...
Quote:
Originally Posted by nohal View Post
VERY good question. Of course it would be the best and most accurate, but I really fear that for practical reasons we will have to calibrate the preprocessed 50% charts (if 50% is the size we decide is the one we will use for our charts).
What reasons do you mean by "practical reasons"?
Maybe it would be possible to just copy small parts of the edges from the fullsize image for the calibration?

Gunther
__________________
Deutschsprachige Community- und Support-Website unter OpenCPN.de
Netsurfer is offline   Reply With Quote
Old 14-07-2011, 00:45   #218
Registered User
 
LeaseOnLife's Avatar

Join Date: Apr 2008
Location: Around once, back to work, for a while :-(
Boat: Sailboat
Posts: 967
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by Netsurfer View Post
Maybe it would be possible to just copy small parts of the edges from the fullsize image for the calibration?
How do I get the correct pixel count for a REF point on an image which doesn't have all pixels from X0/Y0 ?

Different question:
Would a cal tool via the web be too hard to code, too hard for the server to handle the image processing?
__________________
LeaseOnLife is offline   Reply With Quote
Old 14-07-2011, 00:48   #219
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 3,903
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by Netsurfer View Post
Pavel ...

What reasons do you mean by "practical reasons"?
Maybe it would be possible to just copy small parts of the edges from the fullsize image for the calibration?

Gunther
I don't know how on your machines, but on my core i7 wit 3 gigs of ram, even opening any of the full size images takes ages in anything but Picasa.
Also Mapcal does not support JPG, so you would have to convert the fullsize image to PNG or BMP which again will take tons of time.
All this and more seems pretty impractical to me Of course like always, if there's a good reason not to go with scaled down version, I will be easily convinced.

Pavel
__________________
nohal is offline   Reply With Quote
Old 14-07-2011, 00:53   #220
Registered User
 
Netsurfer's Avatar

Join Date: Jan 2010
Location: Cologne, Germany
Boat: Beneteau Oceanis 331
Posts: 557
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by LeaseOnLife View Post
How do I get the correct pixel count for a REF point on an image which doesn't have all pixels from X0/Y0 ?
If you know (and you know) the original size and the coordinates of the part you copy than you know the pixel coordinates for the original by any given coordinates of the copied one.

Quote:
Different question:
Would a cal tool via the web be too hard to code, too hard for the server to handle the image processing?
What would be the advantage of doing so?
All you need (for the "normal" rectangular charts with given position for the SW and NE corners) is a graphic program where you can get the according pixel coordinates from.

Gunther
__________________
Deutschsprachige Community- und Support-Website unter OpenCPN.de
Netsurfer is offline   Reply With Quote
Old 14-07-2011, 00:58   #221
Registered User
 
Netsurfer's Avatar

Join Date: Jan 2010
Location: Cologne, Germany
Boat: Beneteau Oceanis 331
Posts: 557
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by nohal View Post
I don't know how on your machines, but on my core i7 wit 3 gigs of ram, even opening any of the full size images takes ages in anything but Picasa.
Also Mapcal does not support JPG, so you would have to convert the fullsize image to PNG or BMP which again will take tons of time.
All this and more seems pretty impractical to me Of course like always, if there's a good reason not to go with scaled down version, I will be easily convinced.
First of all I don't see why we should use MapCal.
Second about the speed: Have you tried to use the GD Lib with PHP?
I am using it to stitch the downloaded tiles. It takes less than a minute to stitch 4000 tiles with 200 MB. I guess (though not tried yet) it should be able to copy the 4 corners (let's say each 512 x 512 px) pretty fast.

Gunther
__________________
Deutschsprachige Community- und Support-Website unter OpenCPN.de
Netsurfer is offline   Reply With Quote
Old 14-07-2011, 01:09   #222
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 3,903
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by Netsurfer View Post
First of all I don't see why we should use MapCal.
Second about the speed: Have you tried to use the GD Lib with PHP?
I am using it to stitch the downloaded tiles. It takes less than a minute to stitch 4000 tiles with 200 MB. I guess (though not tried yet) it should be able to copy the 4 corners (let's say each 512 x 512 px) pretty fast.
Gunther
Gunther,
there's no problem to cut the 4 corners with 4 imagemagick calls taking no time of course. The problem is that this is a perfect approach just for a "perfect" chart. I really don't know what's the percentage of such charts in the portfolio and how bad are the others. I need to see numbers and do some trials to be able to make qualified decisions, the thoughts and feelings are not enough.
No need to use Mapcal if we have something better people can intuitively use. Right now we don't. Once the online header generator is ready with the appropriate instructions to use it, we have no problem.

Pavel
__________________
nohal is offline   Reply With Quote
Old 14-07-2011, 01:21   #223
Registered User
 
Netsurfer's Avatar

Join Date: Jan 2010
Location: Cologne, Germany
Boat: Beneteau Oceanis 331
Posts: 557
Re: Charts II: NGA - 2700 Charts

Pavel
Quote:
Originally Posted by nohal View Post
The problem is that this is a perfect approach just for a "perfect" chart. I really don't know what's the percentage of such charts in the portfolio and how bad are the others. I need to see numbers and do some trials to be able to make qualified decisions, the thoughts and feelings are not enough.
That's the point. Because I got the impression that you are already suggesting other methods ... "I really fear that for practical reasons we will have to calibrate the preprocessed 50% charts"!
Quote:
No need to use Mapcal if we have something better people can intuitively use. Right now we don't. Once the online header generator is ready with the appropriate instructions to use it, we have no problem.
Sorry, but what does any header generating tool has to do with the calibration?
First we need a method to get at least 2 (better 4) REF points. That means we need the pixel coordinates and the according lat/lng values from each chart image. The most accurate way of doing this would be to use the fullsize image. So before thinking about any other way we should have a look if there will be a practical solution for this. And only if not, then we have to look for alternatives. That's my opinion.

Gunther
__________________
Deutschsprachige Community- und Support-Website unter OpenCPN.de
Netsurfer is offline   Reply With Quote
Old 14-07-2011, 01:42   #224
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 3,903
Re: Charts II: NGA - 2700 Charts

Quote:
Originally Posted by Netsurfer View Post
Pavel

That's the point. Because I got the impression that you are already suggesting other methods ... "I really fear that for practical reasons we will have to calibrate the preprocessed 50% charts"!

Sorry, but what does any header generating tool has to do with the calibration?
First we need a method to get at least 2 (better 4) REF points. That means we need the pixel coordinates and the according lat/lng values from each chart image. The most accurate way of doing this would be to use the fullsize image. So before thinking about any other way we should have a look if there will be a practical solution for this. And only if not, then we have to look for alternatives. That's my opinion.

Gunther
Gunther,
things without context look so nice and easy
By the header generation tool I of course meant something, that (at least) stores the input REFs & PLYs and does at least a formal check of the input etc.
All this is a natural part of Mapcal and we have currently no replacement for that (apart from a totally manual approach). Not talking about Mapcal's intuitivity for anybody used to use it, which is of course questionable.
With Mapcal, using the fullsize images is almost impossible though. That's a fact. There's a question: Is the tradeoff in accuracy if we use scaled down images with Mapcal acceptable or not? I really don't know as I didn't try yet. We are just discussing it to find the best way ahead.

Pavel
__________________
nohal is offline   Reply With Quote
Old 14-07-2011, 02:01   #225
bcn
Registered User

Join Date: May 2011
Location: underway whenever possible
Boat: Rangeboat 39
Posts: 2,232
Re: Charts II: NGA - 2700 Charts

As proposed by THomas test of transformation/size reduction starting with jpg-->png first
All in GIMP. First value with png quality 5/ second with quality 0 (lower is better)

Original 100% 53093.jpg 13.020kB

PNG 100% 21.323kB 375.151kB

PNG 50% px 6.676kB 93.793kB

PNG 32colors 1.326kB 31.269kB

The result in that workflow at a zoom of 200% is slightly better when doing reductions in jpg. Very subtle differences. The improvement between quality 5 and 0 at 200% is only visible with imagination.
No filters applied.

Hubert
__________________

__________________
bcn is offline   Reply With Quote
Reply

Tags
charts

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off



Our Communities

Our communities encompass many different hobbies and interests, but each one is built on friendly, intelligent membership.

» More about our Communities

Automotive Communities

Our Automotive communities encompass many different makes and models. From U.S. domestics to European Saloons.

» More about our Automotive Communities

Marine Communities

Our Marine websites focus on Cruising and Sailing Vessels, including forums and the largest cruising Wiki project on the web today.

» More about our Marine Communities


Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 20:51.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2016, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2016, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.