Charts II: NGA - 2700 Charts

[r.fairman]

sorry where is the newest version of imgkap as a txt file for header would not work for me. Certainly using .txt as a header file would overcome the danger of overwriiting existing .kap files. I'm using v 1.11

regards

richard

[LeaseOnLife]

Originally Posted by r.fairman

sorry where is the newest version of imgkap as a txt file for header would not work for me. Certainly using .txt as a header file would overcome the danger of overwriiting existing .kap files. I'm using v 1.11

regards

richard

Newest version is 1.11

http://www.dacust.com/inlandwaters/imgkap/

It doesn't accept the mc2bsbh output *.hdr. But when I rename the *.hdr to *.txt it works:

imgkap -n 700-Port-Elizabeth-To-Madagascar.png 700-PORT-ELIZABETH-TO-MADAGASCAR.txt

[r.fairman]

ok retested and works when header file is .txt. must have had a senior momemt. Hope the senior moments don't all join up

At some stage we should document the convert to .kap procedure so we all folow the same route, or similar routes and produce consistent compatible results, Once things settle down and the route to .kap is finalised tested and set in stone i am happy to document for approval and distribution

regards
richard

[NavPak ECS]

<< I would like to keep compatibility to SeaClear, Fugawi, RayTech, Maxsea.... >>

You might want to check the IFM of the finished Kaps. If it is 7, then you may have compatability problems. I think the program that makes the KAP assigns the IFM, or can you specify it in the header before conversion?

There are literally hundreds of BSB readers, and I can see how many of the developers would conclude that nobody would ever make an IFM 7 chart, so they did not implement it, just as most developers would assume that nobody would make a 63 color image of a 5 color chart.

Im wondering if the JPG to BSB conversion results in IFM 7. In other words, once the image is thrown off the cliff of clarity into the pit of lossy random noise, can it be saved or is it destined to be an IFM 7 BSB that may choke on some programs? Or will it take too much manual processing to get it under 16 colors?

Just a wild guess. If I get time, I will take a look at it.

[r.fairman]

again a gteat post,but what is IFM and how do i check it m

my path to kap was

straight conversion jpg tp png using imagemagick

calibrate using Mapcal to create chartcal.dir

Convert chartcal to hdr files using mc2bsbh

change .hdr extensions to .txt

created .kap chart using imgkap with the switch - colors 127

richard fairman

[LeaseOnLife]

Originally Posted by r.fairman

again a gteat post,but what is IFM and how do i check it

IFM/7 in the header: The output of mc2bsbh doesn't have it, but after processing with imgkap, this IFM/7 is written into the kap file. Yes, I do think reducing to 16 colors is too much manual work, error-prone, non-repeatable, can not be crowd-sourced.

But is this really the problem? We have opencpn, seaclear, fugawi working so far. Would be great to hear some tests on some other major programs.

[r.fairman]

Ok I know google it

Found our IFM = color depth per pixel
I opened my kap file in word and found IFM = 7
I will try again and load into fugawi and coastal explorer to test compatibility

regards

Richard

[r.fairman]

Here is new compatibility list using the -colors 127 in imgkap to create .kp file

Compatible:
SeaClear
OpenCPN
KapView
Photoshop
FUGAWI bsb viewer

Not compatible
Coastal navigator- message invalid bsb I can load into the chart portfolio and the thumbnail diplays when chart is clicked on but will not display
Fugawi 45 will not import depite using the bsb non maptech option
Offshore navigator Will not load
Polar Navy will not lload in chart manager any comments brak
Paytech navigator v 6.0 detected on search and ays installing but will not display

Not tested
Maxsea

[LeaseOnLife]

Originally Posted by r.fairman

my path to kap was

straight conversion jpg tp png using imagemagick

calibrate using Mapcal to create chartcal.dir

Convert chartcal to hdr files using mc2bsbh

change .hdr extensions to .txt

created .kap chart using imgkap with the switch - colors 127

My trials have come up with a very similar process:

1. I create a working directory for a few charts only, named similar to the chartnumbers (example 6158x) and copy the full size images into this dir. Purpose: keep the CHARTCAL.DIR small and keep the converted images and header files for future improvements/corrections and easy rebuilding of the kap files
2. in this working directory, I first convert the full size images to a quarter size png, I use nconvert (and xnview from the same author):
   nconvert -resize 50% 50% -out png filename.ext
   careful: it overwrites any file named filename.png
   I remove the original full size file from this working dir, I keep the full size files organized somewhere else
3. Use xnview to rotate the images and split out and crop any inset charts, saving them as separate pngs: 61582-A, 61582-B ... Keep xnview running to read chart info in next step
4. using MapCal, enter chart info, calibrate each image, define borders.
   I understand, the hdr can be created manually, but this is NOT a user friendly option.
5. mc2bsbh to create the *.hdr files for each chart
6. rename *.hdr files to *.txt
7. use imgkap to create kap files, use -n option to reduce to 127 colors for compatibility reasons
8. copy kap files to my BSB chart directory
9. in opencpn/setup/charts: "force rebuild of database"
10. review charts, use quilting to see how new charts fit with other charts

Have a look at the results, here is a copy of the working directory (Reunion with a bunch of harbor charts):

http://www.4shared.com/folder/PQVExE...rking-Dir.html
Please have a look and comment


Besides process improvements and additions, some of the tools need improvement also:

• mc2bsbh: use comments out of MapCal/CHARTCAL.DIR ("CR" field) and fill field(s) in *.hdr. Possibly needs some checking of length, multiline, valid characters... I don't think we should add another tool to add info about the charts (Edition, date, ...)
• imgkap: use *.hdr files by default, then the renaming step is not necessary
• imgkap: use -n option by default to make charts more compatible

[LeaseOnLife]

Originally Posted by r.fairman

Not compatible
Fugawi 45 will not import depite using the bsb non maptech option

Richard, can I get a kap file to test it here?

... if you would use a 4shared account, we could see/copy/improve each others files...

[NavPak ECS]

<< IFM = color depth per pixel >>

That is true for IFM 1 through 6. IFM 7 can have unlimited BPP. IFM 1 through 6 uses a combined color\repeat bitstream ( sorry some of my keys dont work, like apostrophe and forward slash) with a fixed size color field, and IFM 7 uses seperate color and repeat bitstreams for an unlimited color field size.

According to the original NOS tech paper, the unlimited color field size could allow smart rasters, where bits could be included in the color stream for GIS or danger flags in anticipation of ANCS II. That program was deemed as too expensive to maintain so it was scrapped, thus IFM 7 is dead. The tech paper describes IFM 7 as a different format, rather than a variation of the same format, so this is why I think that many software developers probably did not bother to support it.

Also according to the paper, the purpose of the NOS compression scheme was to minimize the number of charts that would need to be spanned across more than 1 floppy disk. Excess colors defeats the purpose of the NOS format (BSB) so IFM 7 charts were probably never made by a mainstream source. To correctly implement IFM 7, the production software needs to determine which colors are more important and put those first in the palette.

Im thinking that the freeware that produces KAPs from JPGs is overwhelmed by the colorful JPG noise and can not determine which colors are most important for palette ordering so it is forced to use a huge palette with no order and cobble together an IFM 7 image; just another wild guess.

PNGs are OK, but Im afraid that once a chart image is thrown into the JPG pit, it may be difficult to bring it back for a commercial quality chart.

It would be interesting to know if the freeware KAP generators make an IFM 7 from a PNG, Packbits TIFF, or other RLE image, and also how they respond to the same image at different color depths. If I get some free time, I will take a look at it.

[Netsurfer]

Hi folks,

before going any further in the details of the header and kap creation I would like to announce the start of OpenCPN.info which is meant as a place to coordinate our efforts.

Once you registered an account you can post your links as a comment on the respective site. I'll then add them to the DB.

One more please: Hosting files at 4shared is really a pain for downloading. Sometimes one has to wait 360 seconds before the link appears. Therefor I really recommend the use of Dropbox. It makes things a lot easier ... .
And if you have no Dropbox account yet and like to do me a favour, please use the following link to register one: http://db.tt/S1mKkbs
By using this link for registering your account I'll get some more space (to host more of the NGA charts).

Thanks
Gunther

[cagney]

Everyone using MapCal and mc2bsbh, in the end you have to check your header files manually. To test on other software we need the headers as complete as possible. Who knows what these software are checking in the headers?
Just an example. OpenCPN is happy without the line VER/XX, but I think Fugawi throws a fit.
Thanks NavPak ECS for the info about IFM. Interesting!
I see IFM 7 quite often. It's pointless to specify in the header as, at least tif2bsb overwrites the value. I suspect imgkap does the same.

Here is Richards header from earlier in the thread.

! Created by mc2bsbh beta07 - Use at your own risk!
VER/2.0
BSB/NA=Bodrum Harbour,NU=
    RA=7286,5763,DU=179
KNP/SC=10000,GD=WGS84,PR=MERCATOR,PP=UNKNOWN
    PI=UNKNOWN,SP=UNKNOWN,SK=0.0,TA=90.0
    UN=UNKNOWN,SD=UNKNOWN
    DX=1.42,DY=1.42
REF/1,6977,359,37.0356667,27.454
REF/2,252,5465,36.97,27.35
PLY/1,37.0402172,27.3461802
PLY/2,36.9662301,27.3461802
PLY/3,36.9662301,27.4587013
PLY/4,37.0402172,27.4587013
DTM/0,0

NU is missing -> simply the chart number. NGA 27120, for example
PP is never unknown. Sometimes it is given on the charts, otherwise use mid latitude for Mercs and mid longitude for TMercs
UN is never unknown.
SD is hardly ever unknown, a few NGA charts refer to "local datum" though, and gives you no option but to use "unknown".

REFs. I don't know, but it's possible that some software don't like just 2 ref points. I don't think I have see a commercially produced chart with just 2,.. 3 is probably the minimum I have seen.

A question, as I don't use mapcal. On the NGA charts, 4 positions are given very accurately, the 4 corners. That is why I, as a minimum uses these points as REFs. A simpple copy/paste then gives the PLYs. I use Gimp at 200% to get the pixel-count for these positions. Can you obtain the same accuracy in MapCal?

Thomas

[idpnd]

Originally Posted by Netsurfer

before going any further in the details of the header and kap creation I would like to announce the start of OpenCPN.info which is meant as a place to coordinate our efforts.

Once you registered an account you can post your links as a comment on the respective site. I'll then add them to the DB.

Superb!

[LeaseOnLife]

Originally Posted by cagney

A question, as I don't use mapcal. On the NGA charts, 4 positions are given very accurately, the 4 corners. That is why I, as a minimum uses these points as REFs. A simpple copy/paste then gives the PLYs. I use Gimp at 200% to get the pixel-count for these positions. Can you obtain the same accuracy in MapCal?

Thomas

Thomas, not everybody has, or wants to use gimp. I am trying to come up with a way where some more people could help out doing calibrations. As with colorspace, reduction of colors, working with color pallets: Is this something a normal user wants to do for more than maybe the crucial charts the user wants?

If we make it reasonably easy, more users will be willing to chip in some time and get a few more calibrations done.

Netsurfer:

Hosting files at 4shared is really a pain for downloading.

I am at almost 6GB for 66% of Region 6 charts, full size. This would fill 3 free dropbox accounts. I like dropbox for other purposes. Get a free account (10GB) on 4shared, use webdav or the desktop program and there will be no waiting time. Sharing between users is about as easy as with dropbox.