Chart Downloader/Updater Plugin

[rgleason]

My Vote is for Coast Guard Districts!
Pavel, may I upload to Opencpn Beta Windows Plugins the current Chartloader Plugin? or do you want to wait for a few more changes?

[transmitterdan]

Originally Posted by nohal

transmitterdan...
Read the README When you get to "Windows Specific Libraries" you will soon know where to get it... I will not include the binary build dependencies in the source tree, it is a waste of space and bandwidth.

Pavel

Pavel,

Still something is wrong when I build the chartdldr_pi plugin in Windows. This is the log file from O.

[transmitterdan]

Originally Posted by rgleason

My Vote is for Coast Guard Districts!
Pavel, may I upload to Opencpn Beta Windows Plugins the current Chartloader Plugin? or do you want to wait for a few more changes?

Rick,

My vote is to wait just a bit. We still have bugs in the plugin (COE rivers doesn't work). And it is still cumbersome for newbies. Let's see if we can polish it a bit. I worry that newbies will find it complex to use and not see the benefit right away. For experienced users it is great but they already know how to do this stuff anyway.

Dan

[SethDart]

Hi,

Please let me explain a little bit more the rationale for the directory structure we're using.

First, this tool may (and I really hope so) be used for others sources, but for now, it's built over to fit the needs of NOAA chart distribution.

Before this tool, you'd manually download a NOAA chart set and extract it. Then use it.

For this example, I'll use Nevada (NV) as it's a small set:
http://www.charts.noaa.gov/RNCs/NV_RNCs.zip
Extract it, you'll end up with a structure similar to this :

BSB_ROOT/
  18665/
    18665.BSB
    18665_1.KAP
  18687/
    18667.BSB
    18667_1.KAP
    18667_2.KAP
    18667_3.KAP
    18667_4.KAP
    18667_5.KAP
    18667_6.KAP

If you download several states / districts / regions or the whole set, you'll end up with all charts in BSB_ROOT

With this tool, we try to fit this structure.

So if you want to use this plugin to manage your chart set, you can add a catalog for "VA RNC" pointing it to our BSB_ROOT folder. It'll find the charts and tell you if there's any update or new chart. When you update it or install any new chart, they'll be installed in a location that match the NOAA directory structure.

If you've several catalogs, you can repeat this and check each catalog one by one. Or you can use the "All RNC" catalog and it'll work as well. You'll see your charts an be able to update it or add any new chart.

That's exactly the same with ENC (in that case, the shared folder will be ENC_ROOT)

If you want to extend your navigation zone to California for example, you can add the appropriate catalog and install the charts with this tool. Or you can download the whole CA zip, extract it and use this tool to update charts afterwards.

[SethDart]

If you want a separate directory for each state / district / region, you can create a NV folder and add a catalog going to this folder.
This will work fine until you want to use a larger catalog (or All RNC) Then, you'll not be able to update all at once. That's why I'd recommend keeping the original directory structure.

You can name the first level folder the way you want (the original BSB_ROOT or ENC_ROOT) we strip that name when extracting charts. But wee keep the remaining structure.

Hope I made it clearer ;-)

[rgleason]

SethDart I think that structure/operation works for me and I hope it is clear to New users. I do use Coast Guard Districts as I found that more manageable.

It might be good to put some of the right button menus as buttons for new users, but I found them very easily, ..did not realize the first time, but did do right button menus after Browse for destination directory and it works.

Pavel, I will wait for user improvements to upload....

This thread really moves along.
I am attaching my log files in case they may be helpful. Windows7.
The small one uploaded ok, please take doc off the other one it is zippped.

[transmitterdan]

SethDart,

First, thank you for help getting this plugin up and running. I fully agree we should keep the future in mind. I understand your logic in choosing the directory structure. The issue I have with it is that it makes creating groups very time intensive and not obvious. When all BSB or ENC charts are in one root folder it's not easy to see which charts belong in a group. Many areas require hundreds of charts to have a regional group (e.g. east coast of Florida).

I am thinking more about the new O users. The real audience for this plugin is the newbie in my view. They just want to make a couple of mouse clicks and see their charts. The mouse clicks should be obvious by looking at the downloader tab. They don't know anything about the directory structure of charts or even what BSB_ROOT is. They just want to see charts for where they are planning to go.

US Regions is probably the most likely grouping that the new user will want. There are not too many regions so that would make it easier to navigate the drop-down than having all the states listed.

With regard to automatic updating I think most new users will be happy to click a button and have all their charts updated. They don't want to think about the directory structure. I agree with you that most users will not want to have to make lots of mouse clicks to update all the charts. I would guess some sort of search from a top level directory looking for all the .xml files and then automatically downloading new or updated charts into each respective sub-folder would make 99% of users happy. I can't think why any O users would want to keep charts that are out of date. If some very experienced user wants to update some charts and keep others that are out of date they can do that manually now without this plugin.

[transmitterdan]

Originally Posted by bcn

Dan,

when tackling the issue of the charts folders, please take the following environment into account (this is from a discussion we had with Dave during the S-63 implementation):

Thank you for telling us about that. I think what we are doing is no different than manual downloading and unzipping the files. So whatever folder one would manually put the files in can be selected for the new downloader. I think it would be sufficient if there is to be a default location then it should pay attention to the .ini file redirection.

[nohal]

I have put the stuff that appeared so far into https://github.com/nohal/chartdldr_pi/issues - feel free to comment and (better ) send patches resolving it...
If I forgot something, create new issues. Atomic tasks, clearly described, if possible, please...

Pavel

[transmitterdan]

Patel,

I think it is working as you intended except for maybe the inland rivers. If you need help with that I can dig into it tomorrow. The Corp. of Engineers web site is a disaster right now. I am not sure when they will get it fixed. Apparently they are trying to move it out to the cloud and many of their links are broken. For now my opinion is we can leave that off until we know how they will work their web site.

The regions in the drop-down list make it work very nice. I think we can go to beta testing as soon as you are ready. But before we put it up for general download we need a little tutorial for newbies on how to use it. Has anyone built it for OS-X? I have tested it a lot today on Linux and it worked very well.

The only issue I ran into today was some charts did not download the first time (about 3% failure rate). A second try would always get all charts. I had the same problem using curl at the command line. I "fixed" it by giving curl the retry option with the retry count of 2. I don't know how to do that in wxCurl.

it was easy to put regions into groups as SethDart explained. I really like it so far. It's a huge step in just a few days.

[rgleason]

Recent compile, just a few errors. Actually one.


Build succeeded.

"C:\Data-Dart\Up-Soft\Navigation\o-plugin\chartdldr_pi\build\ALL_BUILD.vcxproj"
(default target) (1) ->
"C:\Data-Dart\Up-Soft\Navigation\o-plugin\chartdldr_pi\build\chartdldr_pi.vcxpro
j" (default target) (3) ->
(ClCompile target) ->
c:\data-dart\up-soft\navigation\o-plugin\chartdldr_pi\src\tinyxml/tinyxml.h(77
4): warning C4251: 'TiXmlNode::value' : class 'std::basic_string<_Elem,_Traits,_
Ax>' needs to have dll-interface to be used by clients of class 'TiXmlNode' [C:\
Data-Dart\Up-Soft\Navigation\o-plugin\chartdldr_pi\build\chartdldr_pi.vcxproj]
c:\data-dart\up-soft\navigation\o-plugin\chartdldr_pi\src\tinyxml/tinyxml.h(89
...Deleted some

10 Warning(s)
0 Error(s)

[transmitterdan]

These warnings about dll-interface just started showing up for me in all the plugins since this weekend. I don't remember seeing them before that.

[rgleason]

I just recompiled and it worked fine. 0 warnings 0 errors. Don't know what it was. Rebooted computer last night.

I've been checking out the changes made to NOAA Regions.
Selected Noaa Region 02, pointed the destination to BSB directory and then selected about 1/2 the charts and then downloaded. They were downloaded via wifi in a very short time. The charts downloaded directly to BSB in a flat directory format (without regions).

Since I like to keep my Noaa charts by Noaa Region to be able to enable and disable logical groups of charts in the Chart Directory, I would prefer to download Region 02 charts to 'Noaa02" directory for example [right now I am using format "Region 01"]. I try to keep the name short for efficiency and now I've decided to eliminate spaces because that takes extra characters too.

To do this, I found that I could Browse to the destination folder, in my case,
c:/data-dart/BSB_ROOT/, then right click and select "New Folder", type the new name for the folder "Noaa02" hit enter. Then using the Destination Browser, click on "Noaa02" and hit enter to set the destination for these Noaa Region 02 charts.
From there I can select the charts and then download. They land in the new folder created "Noaa02".

The only thing I notice at this point is there is no way to "Edit" a Catalog to Destination entry. Perhaps we could have a right click ability to edit a single highlighted catalog entry?

I checked the IENC and it seems to be a work in progress with this being the main

Launch page
Army Geospatial Center > Missions > Echarts > Product Downloads

with the Download button going to amazon cloud server here
Home

and this is a summary of the Rivers covered
Army Geospatial Center > Missions > Echarts > Inland Electronic Navigational Charts

[SethDart]

Hi,

Rgleason, you can do that with the plugin:
In BSB_ROOT, you create a folder named "Region 01"
Then you add the region in the plugin using "Region 01" as the destination.
It'll work as expected and you can update your charts there.

Unfortunately, you can't use BSB_ROOT as the destination for this catalog because when updating a chart, they are extracted in BSB_ROOT, not "Region 01". In a future version, we may find a way to stick to local structure. But it's not yet available. Patch welcome... ;-)

Edit is a feature request: https://github.com/nohal/chartdldr_pi/issues/11

Enjoy!

[rgleason]

SethDart,
I made a new folder from setting the destination browser by "right click" and select "New Folder". Then typed the folder name, then selected the folder and both the Region02.xml and the Charts downloaded landed there!

Now when I highlight that Catalog there are no chart selections that come up even though the Region02.xml does exist in that directory! Why doesn't the program read from the directory being pointed at?

It should also download to there as well I think.

Downloader Destination Path = Destination Path set by user?
Catalog Reading path = Destination Path set by user?
Maybe there is some other way of expressing this. I have not thought of all the consequences, but it seems to make sense. Would this mean that to update charts you would have to point to various folders?
Would that be too much work?

Thanks for such a great development!