Can't Get ENC to Show

[fgd3]

Originally Posted by dacust

Your second paragraph is a good question, and I'd like to hear the answer, as well. I THINK that rebuilding the database just re-discovers what charts you have. I don't THINK it removes any SENC charts, and I'm sure it doesn't automatically rebuild them.

-dan

If that's the case we would have to delete the SENC directory, then force a full database rebuild after downloading new charts. That strikes me as a serious flaw in the program's design.

Fabbian

[fgd3]

Originally Posted by Sail323

I found many 40,000 scale charts missing for the AICW on my trip south this year. This meant no soundings on ENC charts from Mile 740 to Mile 910 as well as parts of Georgia and South Carolina.

John

I don't think that's the program's fault. I believe NOAA simply hasn't released large scale ENCs for most of the ICW. For the time being it appears that the only way to see soundings along the ICW is to use the RNC small-craft charts.

Fabbian

[cagney]

dacust

After deleting the SENC, next time you try to display that chart, it will be rebuilt from the ENC chart (not next program open).

You could very well be right here, or,.. it could depend on the operating system. If you, on linux, displayed the chart in a session, it's in the memory (or swapped to disk), what will happen then? I have not tested. I said "restart" to get around all this. But it's an interesting subject.

Originally Posted by fgd3

If that's the case we would have to delete the SENC directory, then force a full database rebuild after downloading new charts. That strikes me as a serious flaw in the program's design.

Fabbian

My understanding is that OpenCPN does the right thing when it comes to ENC files corrections and new versions.
Each cell has a base ENC file with the ending .000. Updates to this cell is named .001, .002 etc. The program checks for these updates, probably on startup.

I don't think that the full database rebuild has much to do with it. Dave even thinks it is more or less redundant. It is however necessary, to make minor changes to RNC text header visible. A thing that normal users are not involved in at all. The SENC files are certainly not recreated, that would take way to long time.

These updates/chart corrections, are then integrated in the SENC file.
OpenCPN also checks the case were the base file is updated, in other words a new edition of the chart, not just corrections. The relevant SENC file will then be replaced.

This is a very good subject, that needs to be treated in the documentation. I'll ask Dave for the details later. For the moment he's on Island Time

Thomas

[ktcanuck]

From what you folks are saying it sounds like there is some kind of chart history saved within the OpenCPN structure somewhere. If that is the case, how would one delete it so the database rebuild is completely fresh?

[Sail323]

Originally Posted by fgd3

I don't think that's the program's fault. I believe NOAA simply hasn't released large scale ENCs for most of the ICW. For the time being it appears that the only way to see soundings along the ICW is to use the RNC small-craft charts.

Fabbian

You're right. It is not the program's fault. The ENC charts cover just about all of the ICW from Norfolk, VA to Beaufort, SC, but from there south the coverage gets spotty with a large chunk missing between Mile 740 to Mile 910 in Northern Florida.

I contacted NOAA about this and this was their reply;

"Thank you for your inquiry. We are continuing to evaluate our ENC coverage and produce new ENCs based on priorities and resources. We currently plan to fill out our Florida coverage by the end of 2012."

[dacust]

Originally Posted by cagney

dacust

You could very well be right here, or,.. it could depend on the operating system. If you, on linux, displayed the chart in a session, it's in the memory (or swapped to disk), what will happen then? I have not tested. I said "restart" to get around all this. But it's an interesting subject.

...

Ah, actually I did not state that very well. In the way you took what I posted, your response was accurate. But it's not what I actually meant.

I agree with you that most likely restarting OpenCPN would be required. What I was nitpicking on was that the SENC would not actually be rebuilt at restart time, but at next access time. Very small nitpick that probebly doesn't matter to most people.

As to the rest, if the program is running when the file is deleted, then I totally agree about it working different on different operating systems. On the system I work on (AS/400), it actually behaves differently depending on what type of object you try to delete. If in use, it may not allow you to delete it, or the program may error out with a "Tried to refer to part or all of an object that no longer exists", or it may continue on working with the object in memory. Depends on the object type.

I agree. Interesting subject.

EDIT: Oh, and your explanation of the .000, .001, etc. clears up my confusion about what happens when you get updated charts. Thanks.

-dan

[fgd3]

Originally Posted by cagney

My understanding is that OpenCPN does the right thing when it comes to ENC files corrections and new versions.
Each cell has a base ENC file with the ending .000. Updates to this cell is named .001, .002 etc. The program checks for these updates, probably on startup.

Thomas

So what happens when a chart is re-issued? NOAA periodically issues charts with a new date. All previous updates are incorporated into the new base chart which has a .000 file name extension.

Fabbian

[fgd3]

What's the purpose of the SENC directory? Why not read the chart files directly from the active chart directories?

Fabbian

[cagney]

Originally Posted by fgd3

So what happens when a chart is re-issued? NOAA periodically issues charts with a new date. All previous updates are incorporated into the new base chart which has a .000 file name extension.

Fabbian

It's of course up to the user to download the upgrade to the new edition. OpenCPN is not messing with your file structure, that's up to you.

OpenCPN compares release dates to see if a new .000 release is newer than the date, that the present SENC file is based on (date of the latest correction). If this is the case, the SENC files are replaced.

What's the purpose of the SENC directory? Why not read the chart files directly from the active chart directories?

Remember that ENC charts are based on an international standard that OpenCPN intends to follow. This is not an OpenCPN specific issue. All navigation programs create SENC files, because chart files, as they are, cannot be processed in an efficient way, directly.

Note from one of the many international committee metings...

► In order to get efficient data structures that facilitate the rapid display of ENC data,
ECDIS firstly converts each ENC from S-57 ENC format into an internal format
called SENC – System ENC – which is optimised for chart image creating routines.

The SENC format is internal to each navigation program, and not standardized. One issue that may come up later on as OpenCPN develops is "Can the present SENC format be improved?".

Dacust, you are of course correct in your little "nitpick", that is how it works.

ktcanuck As you can see the chart database is never really "stale", and it is not a good idea to rebuild it, for the reason that it takes a bit of time to build each SENC.
But there could be legitimate reasons to rebuild the chart database.One cannot, for example exclude the possibility of a corrupt database, if for some reason there is a crash when building a SENC.
For the locations of these files check Installing OpenCPN | Official OpenCPN Homepage. Scroll down to the heading "Location of Important Files. Log and config files."
The SENC and CM93 directories and the the file, chartlist.dat, together makes up what could be called "the chart database".

Thomas

[fgd3]

Originally Posted by cagney

It's of course up to the user to download the upgrade to the new edition. OpenCPN is not messing with your file structure, that's up to you.

OpenCPN compares release dates to see if a new .000 release is newer than the date, that the present SENC file is based on (date of the latest correction). If this is the case, the SENC files are replaced.

That reassures me that when I download a new chart OpenCPN will be cognizant of it and won't rely on the cached old version.

Originally Posted by cagney

Remember that ENC charts are based on an international standard that OpenCPN intends to follow. This is not an OpenCPN specific issue. All navigation programs create SENC files, because chart files, as they are, cannot be processed in an efficient way, directly.

Thanks for the explanation. I'm sure the SENC files are saved for performance reasons as it would take a long time to rebuild them every time the program was opened or a chart was viewed.

I'm still unclear on what "Force Full Database Rebuild" does. I would expect it to scan all the active chart directories and rebuild all the SENC files. I know it takes a long time, but it seems to take just as long if I remove active chart directories and then add them back. Could you explain that for me? Thanks.

Fabbian

[bdbcat]

Hi guys and gals....

A drive by posting from near Nassau....

1.Back to the OP issue of missing charts depending on the dir structure.
What I need is two log files:

a. Correct operation with 4 subdirectories specified individually to Toolbox->Charts.

b. Incorrect operation with parent directory only specified to Toolbox->Charts.

I both cases we need the Rebuild Database option selected.

I the failure case, we need to know which charts are then unable to open.

The logfiles will be huge. Zip and mail to me if more convenient.

Thanks
Dave

[bdbcat]

Hi....

More discussion on SENC file update logic.

SENC files are declared stale and regenerated under the following conditions:

1. .000 file EDTN field is newer than the EDTN data recorded in the SENC
2. or, If there are any .001, .002, etc (update) files which have not been processed.
3. or, if the .000 file is "newer" than the SENC file, or the size of the .000 file has changed.

Here is some code from S57chart.cpp, which I hope is understandable...

//              Anything to do?

                                //  SENC file version has to be correct for other tests to make sense
                                if(senc_file_version != CURRENT_SENC_FORMAT_VERSION)
                                      bbuild_new_senc = true;

                                //  Senc EDTN must be the same as .000 file EDTN.
                                //  This test catches the usual case where the .000 file is updated from the web,
                                //  and all updates (.001, .002, etc.)  are subsumed.
                                else if(!senc_base_edtn.IsSameAs(m_edtn000))
                                      bbuild_new_senc = true;

                                else
                                {
                                    //    See if there are any new update files  in the ENC directory
                                    int most_recent_update_file = GetUpdateFileArray(FileName000, NULL);

                                    if(last_update != most_recent_update_file)
                                          bbuild_new_senc = true;

//          Make two simple tests to see if the .000 file is "newer" than the SENC file representation
//          These tests may be redundant, since the DSID:EDTN test above should catch new base files
                                    wxDateTime OModTime000;
                                    FileName000.GetTimes(NULL, &OModTime000, NULL);
                                    OModTime000.ResetTime();                      // to midnight
                                    if(ModTime000.IsValid())
                                          if(OModTime000.IsLaterThan(ModTime000))
                                                bbuild_new_senc = true;

                                    int Osize000l = FileName000.GetSize().GetLo();
                                    if(size000 != Osize000l)
                                          bbuild_new_senc = true;
                                }

                                if(force_make_senc)
                                    bbuild_new_senc = true;

                                if(bbuild_new_senc)
                                      build_ret_val = BuildSENCFile( name, m_SENCFileName.GetFullPath() );

Result: SENC files may be safely ignored by the user, and considered opaque.

Updating the chart database in Toolbox->Charts does not affect the SENC files at all.

SENC files are very large, mainly for performance reasons. We convert area geometries found in ENCs (arrays of lat/lon points) into huge arrays of tesselated triangles, which can be rendered onscreen very quickly. You probably do not want to build SENCs for all of your loaded S57 ENCs unless you plan to actually go there....

Hope this helps the understanding....
Dave

[ktcanuck]

Ok. I will do that tomorrow when I back on the same XP PC that exhibited the problem last time. The Vista one I am on now exhibited no such problem when I went through the exercise just now. Watch the XP one be fine tomorrow just to make look like a dink!!

Keith

Originally Posted by bdbcat

Hi guys and gals....

A drive by posting from near Nassau....

1.Back to the OP issue of missing charts depending on the dir structure.
What I need is two log files:

a. Correct operation with 4 subdirectories specified individually to Toolbox->Charts.

b. Incorrect operation with parent directory only specified to Toolbox->Charts.

I both cases we need the Rebuild Database option selected.

I the failure case, we need to know which charts are then unable to open.

The logfiles will be huge. Zip and mail to me if more convenient.

Thanks
Dave

[ktcanuck]

Originally Posted by ktcanuck

Ok. I will do that tomorrow when I back on the same XP PC that exhibited the problem last time. The Vista one I am on now exhibited no such problem when I went through the exercise just now. Watch the XP one be fine tomorrow just to make look like a dink!!

Keith

opencpn_logs.zip

Interesting difference in size! Also interesting that the effect occurs in an XP and not Vista but then that could be simply the order I have the files and folders.

[bdbcat]

ktcanuck....

Thanks, got the logs. Will research and get back to this thread.

Offline for a few days.

Happy holidays!
Dave