Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Rate Thread Display Modes
Old 26-09-2016, 04:22   #1771
Registered User

Join Date: Nov 2012
Location: Orust Sweden
Boat: Najad 34
Posts: 1,645
Re: Beta Test / Technical

Dave..
PS - PaEC now also successfully tested on a W-XP machine - DS
__________________

__________________
Hakan is offline   Reply With Quote
Old 26-09-2016, 06:57   #1772
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,122
Re: Beta Test / Technical

Quote:
Originally Posted by bdbcat View Post
cagney/hakan...

New code in github, this time with lots of log output to track the SENC build decision tree. Thomas, I hope you may be able to see what is happening as you rebuild SENCs with updates.

I will try the ENC Netherlands set, too.

Thanks
Dave
Dave
Repeating test with renamed updates etc using this mornings git master.
Both PaEC took about 8 min. Afterwards O is still processing charts as I pan around.
Excerpts from the log attached.
PaEC-log.txt.pdf
/Thomas
__________________

__________________
cagney is offline   Reply With Quote
Old 26-09-2016, 07:08   #1773
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 4,492
Re: Beta Test / Technical

Hakan...

Are you able to PaEC for the ENC Netherland set now? I tried it with cm93 on-screen, as you reported. No problem with current github. Took about 4 minutes. Some warnings in the log, no errors.

Dave
__________________
bdbcat is online now   Reply With Quote
Old 26-09-2016, 07:18   #1774
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 4,492
Re: Beta Test / Technical

Thomas...

Still some trouble here. It is not picking up the updates correctly. A logfile entry for a cell with updates should look like this:

Code:
20:22:19 EDT: Preparing to apply ENC updates, target final update is   12.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.001.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.002.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.003.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.004.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.005.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.006.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.007.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.008.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.009.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.010.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.011.
20:22:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US5MD21M.012.
I will investigate on a virtual 64 box.

Dave
__________________
bdbcat is online now   Reply With Quote
Old 26-09-2016, 07:59   #1775
Registered User

Join Date: Nov 2012
Location: Orust Sweden
Boat: Najad 34
Posts: 1,645
Re: Beta Test / Technical

Quote:
Originally Posted by bdbcat View Post
Hakan...

Are you able to PaEC for the ENC Netherland set now? I tried it with cm93 on-screen, as you reported. No problem with current github. Took about 4 minutes. Some warnings in the log, no errors.

Dave
My report here: Beta Test / Technical was for all my ENCs NL included and CM93 on screen. And resent Github

A "Funny" observation: If I run PaEC with raster charts on screen O switch to vector, CM93, after finalized PaEC.

Håkan
__________________
Hakan is offline   Reply With Quote
Old 26-09-2016, 22:32   #1776
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 4,492
Re: Beta Test / Technical

cagney...

New code in github correcting missing log messages. I don't think this actually fixes the problem, though.

Anyway, more log to read and think about.
After you do PaEC once, then again, the log should be instructive.
Something like this, maybe?

Code:
00:28:19 EDT: S57chart::Checking SENC file: /home/dsr/.opencpn/SENC/US2EC03M.S57
00:28:19 EDT:     Rebuilding SENC due to incremental cell update.
00:28:19 EDT: last update recorded in SENC: 9   most recent update file: 10
00:28:19 EDT: Preparing to apply ENC updates, target final update is  10.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.001.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.002.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.003.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.004.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.005.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.006.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.007.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.008.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.009.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.010.
Dave
__________________
bdbcat is online now   Reply With Quote
Old 27-09-2016, 00:18   #1777
Registered User

Join Date: Nov 2012
Location: Orust Sweden
Boat: Najad 34
Posts: 1,645
Re: Beta Test / Technical

Dave
Observations:
1
When I run PaEC a second time the log say:
"07:51:54: ParseAllENC() count = 743"
But there are 1466 files after the first run?

2
For the BE_INLAND files a lot of warnings appear in the log, like this chart:
Code:
08:01:06: S57chart::Checking SENC file: C:\ProgramData\opencpn\SENC\7W7GENO3.S57
08:01:06:     Rebuilding SENC due to missing SENC file.
08:01:07:    Warning 1: For feature "SMCFAC", attribute "DATSTA" ignored, not in expected schema.

08:01:07:    Warning 1: Illegal feature attribute id (ATTF:ATTL[3]) of 17106
on feature FIDN=1437142426, FIDS=212.
Skipping attribute, no more warnings will be issued.
08:01:07:    ENC contains no useable M_COVR, CATCOV=1 features:  C:\Users\admin\Documents\Charts\ENC\BE_INLAND\Kanaal Gent-Oostende\7W7GENO3.000
08:01:07:    ENC contains no M_COVR features:  C:\Users\admin\Documents\Charts\ENC\BE_INLAND\Kanaal Gent-Oostende\7W7GENO3.000
08:01:07:    Calculating Chart Extents as fallback.
08:01:09: S57chart::Checking SENC file: C:\ProgramData\opencpn\SENC\7W5BRUS1.S57
Are these warnings due to mistake from those who made the charts or due to our code?
You could download and check the Belgium inland charts if you like.
Håkan
__________________
Hakan is offline   Reply With Quote
Old 27-09-2016, 05:21   #1778
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,122
Re: Beta Test / Technical

Quote:
Originally Posted by bdbcat View Post
cagney...

New code in github correcting missing log messages. I don't think this actually fixes the problem, though.

Anyway, more log to read and think about.
After you do PaEC once, then again, the log should be instructive.
Something like this, maybe?

Code:
00:28:19 EDT: S57chart::Checking SENC file: /home/dsr/.opencpn/SENC/US2EC03M.S57
00:28:19 EDT:     Rebuilding SENC due to incremental cell update.
00:28:19 EDT: last update recorded in SENC: 9   most recent update file: 10
00:28:19 EDT: Preparing to apply ENC updates, target final update is  10.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.001.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.002.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.003.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.004.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.005.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.006.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.007.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.008.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.009.
00:28:19 EDT:  S57: Applying feature updates from /home/dsr/.opencpn/SENC/US2EC03M.010.
Dave

Dave
I've done the same thing again, using this mornings code.
Both runs were almost identical and took about 8 min each.

See attached file. Note that the first run ends and the second run starts at line 1155.

PaEC-logg2.txt.pdf

Thanks!
/Thomas
__________________
cagney is offline   Reply With Quote
Old 27-09-2016, 10:20   #1779
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 4,492
Re: Beta Test / Technical

cagney...

Once more, please, using this mornings github. More log messages.

We'll get it....

Thanks
Dave
__________________
bdbcat is online now   Reply With Quote
Old 27-09-2016, 11:47   #1780
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,122
Re: Beta Test / Technical

Quote:
Originally Posted by bdbcat View Post
cagney...

Once more, please, using this mornings github. More log messages.

We'll get it....

Thanks
Dave
Dave

Sure you will get it, and I don't mind, while waiting for "Invest 97L" (Matthew) in Port Egmont.

Very fast run, only 8 sec, but nothing much happened.
The log only contained these 2 lines for each run
Code:
13:29:33 AST: ParseAllENC() count = 516
13:29:40 AST: Chart cache purge
O crashed a couple of times, at least once it was when pressing PaEC the second time.

Click image for larger version

Name:	PaECSegfault.png
Views:	24
Size:	41.7 KB
ID:	131903


/Thomas
__________________
cagney is offline   Reply With Quote
Old 27-09-2016, 20:49   #1781
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 4,492
Re: Beta Test / Technical

cagney...

I see another bug. But you can work around it temporarily by making sure you display some (any) ENC or cm93 chart before pressing PaEC.

Thanks
Dave
__________________
bdbcat is online now   Reply With Quote
Old 28-09-2016, 04:36   #1782
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,122
Re: Beta Test / Technical

Dave

Another go. Both runs took about 8 min.
Second run starts at line 1303.

PaEC-logg3.txt.pdf

/Thomas
__________________
cagney is offline   Reply With Quote
Old 28-09-2016, 21:33   #1783
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 4,492
Re: Beta Test / Technical

cagney...

.. the Aha moment arrives...

The trouble with repeated PaEC is that you have duplicates of some cells (in different directories), and those duplicates have different edition/update specifications. So what happens is that these two duplicates "compete" with each other for who gets to create and own the common SENC file. And the SENC file is built over and over, flopping from one cell ownership to the other.

I implemented some code to stop the war, but I'm not sure how we want to handle this case.

Duplicate files in the same directory tree are pruned during addition to the database. But in your case, and mine too, the duplicates are in different trees as specified in options->charts->Chart Files. So they are not pruned. You will see them as duplicate buttons on the chart bar, at certain geographic locations. I think this will be a common situation when using the chart downloader and arranging ENCs by region. The same small scale chart (e.g.US2EC03) will appear in multiple regions. But the two copies of US2EC03 may not have the same edition/update numbers.

My solution is to create the SENC for the latest, most modern, of the two charts, and assign it to both cells. This is a little inconsistent, but at least ensures that the "best" chart will always be chosen.

Maybe we need to implement a "global" pruning step, to absolutely eliminate duplicates? But I recall some discussion where certain chart group architectures wanted to retain duplicates. Ring any bells?

Another alternate solution would be to tie the SENC to the originating cell more closely (by some name hash), so that there may be duplicate SENCs, and not just the single SENC allowed by the current model.

This may require more thought/discussion.

Comments, anyone?

Anyway, new code in github now. Should run fast on repeated PaEC, and not create SENCs as you pan around.

Thanks for your patience.
Dave
__________________
bdbcat is online now   Reply With Quote
Old 29-09-2016, 07:18   #1784
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,122
Re: Beta Test / Technical

Dave

Aha ...good catch

Latest test. Git master from earlier this morning.
As usual I have renamed the last update for three charts in the first round, and restored them for the second run.
It is much quicker now. PaEC now takes about 3.5 minutes.There still seems to be a bit of flip-flopping going on with a few files.It does create a few SENCs as I pan around.
PaEC-logg4.txt.pdf

Quote:
My solution is to create the SENC for the latest, most modern, of the two charts, and assign it to both cells. This is a little inconsistent, but at least ensures that the "best" chart will always be chosen.
I think that this is good enough for now. Test it in the upcoming beta series.
Please mention this issue in the notes for the first beta.

Quote:
Maybe we need to implement a "global" pruning step, to absolutely eliminate duplicates? But I recall some discussion where certain chart group architectures wanted to retain duplicates. Ring any bells?
Very weak bells ...

Quote:
Another alternate solution would be to tie the SENC to the originating cell more closely (by some name hash), so that there may be duplicate SENCs, and not just the single SENC allowed by the current model.
This may be the best solution, but is it necessary?

Thanks

Thomas
__________________
cagney is offline   Reply With Quote
Old 29-09-2016, 07:56   #1785
Registered User
 
transmitterdan's Avatar

Join Date: Oct 2011
Boat: Valiant 42
Posts: 3,132
Re: Beta Test / Technical

Quote:
Originally Posted by cagney View Post
I think that this is good enough for now. Test it in the upcoming beta series.
Please mention this issue in the notes for the first beta.


Very weak bells ...


This may be the best solution, but is it necessary?

Thanks

Thomas
My opinion has evolved on this question. Originally I though all duplicate source chart files would in general be identical. But it appears they are not in all cases.

So it seems we could use either a hashed file name approach or just retain the same directory structure within SENC that the chart folders have (but there are degenerate cases where the latter will fail). We have many cases where there are chart files with duplicate names within various chart file trees. The SENC folder tries to "flatten" this out into a single folder and we end up with the challenge of one SENC file standing in for multiple chart files of the same name.

With a hashed file name the original source file name (minus the full path name) is embedded in the S57 file so it's sort of possible to cross reference the hashed file back to it's source. I worry that this loss of source location will cause problems/confusion even with a hash solution.

I agree that this issue needs some collective contemplation. But it seems to me a bad practice to have cached data that can't be unambiguously linked to the parent data in a bidirectional way. I believe that's a necessary condition to be able to know if each piece of the cache is valid or invalid.
__________________

__________________
transmitterdan is offline   Reply With Quote
Reply

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Beta Marine Diesel michaelmrc Engines and Propulsion Systems 48 23-03-2016 14:44
Need some technical advice....antennas. Just a Tinch Marine Electronics 15 01-12-2007 16:57
Blue Sea Systems Technical Brief GordMay Electrical: Batteries, Generators & Solar 0 16-03-2007 05:16
technical difficulties witchcraft The Sailor's Confessional 1 30-05-2005 15:09
Dow Corning Technical Manual GordMay The Library 0 12-04-2005 17:25


Our Communities

Our communities encompass many different hobbies and interests, but each one is built on friendly, intelligent membership.

» More about our Communities

Automotive Communities

Our Automotive communities encompass many different makes and models. From U.S. domestics to European Saloons.

» More about our Automotive Communities

Marine Communities

Our Marine websites focus on Cruising and Sailing Vessels, including forums and the largest cruising Wiki project on the web today.

» More about our Marine Communities


Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 17:17.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2016, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2016, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.