Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Search this Thread Rate Thread Display Modes
Old 01-07-2018, 03:28   #1
Registered User
 
yachtvalhalla's Avatar

Join Date: Aug 2009
Location: Philippines
Boat: Fuji 32 Ketch
Posts: 849
Chart File Compression

Without finding anything in the manual about compressing chart files ('Compress selected' on the Options/Chart Files Directories page) I have used this to reduce some chart files.


But I find if the chart folder is over 1 GB that the program will freeze, sometimes after several hours of processing.


Is there a buffer or cache limitation somewhere?


Can someone give an explanation of using this feature?


Thanks, Terry
__________________

__________________
VALHALLA's Mooring Page
yachtvalhalla is offline   Reply With Quote
Old 01-07-2018, 05:02   #2
Registered User

Join Date: Jun 2015
Posts: 370
Re: Chart File Compression

Quote:
Originally Posted by yachtvalhalla View Post
Without finding anything in the manual about compressing chart files ('Compress selected' on the Options/Chart Files Directories page) I have used this to reduce some chart files.


But I find if the chart folder is over 1 GB that the program will freeze, sometimes after several hours of processing.


Is there a buffer or cache limitation somewhere?


Can someone give an explanation of using this feature?


Thanks, Terry
Hi,

What is your OS? Is 1GB in one folder or in one with sub folders? Could be a memory leak somewhere.

AFAIK there's nothing special to do, you click it compress and that's all, there's a bug if the size > 2048 Mo though, ratio displayed is wrong (32 bits negative overflow). I triggered this one so > 1GB is possible.
__________________

did-g is offline   Reply With Quote
Old 01-07-2018, 05:59   #3
Registered User
 
yachtvalhalla's Avatar

Join Date: Aug 2009
Location: Philippines
Boat: Fuji 32 Ketch
Posts: 849
Re: Chart File Compression

Quote:
Originally Posted by did-g View Post
Hi,

What is your OS? Is 1GB in one folder or in one with sub folders? Could be a memory leak somewhere.

AFAIK there's nothing special to do, you click it compress and that's all, there's a bug if the size > 2048 Mo though, ratio displayed is wrong (32 bits negative overflow). I triggered this one so > 1GB is possible.

OS is Windows10. The 1GB folder has many sub folders. But compressing a cm93 folder of 559MB (40N to 40S) with its many sub folders worked fine, compressed to 353MB. I haven't tried to compress the full cm93 folder of 1.44GB but will do that tonight.


Appreciate any help, Terry
__________________
VALHALLA's Mooring Page
yachtvalhalla is offline   Reply With Quote
Old 01-07-2018, 16:00   #4
Registered User
 
yachtvalhalla's Avatar

Join Date: Aug 2009
Location: Philippines
Boat: Fuji 32 Ketch
Posts: 849
Re: Chart File Compression

Quote:
Originally Posted by did-g View Post
Hi,
there's a bug if the size > 2048 Mo though, ratio displayed is wrong (32 bits negative overflow). I triggered this one so > 1GB is possible.

Excuse my ignorance but the above means nothing to me except I think you are saying there is a limit to files greater than 2.048GB?


The folder that fails for me is 3.44GB in size.



Also, I was able to compress the full cm93 file from 1.44GB to 910MB.


Terry
__________________
VALHALLA's Mooring Page
yachtvalhalla is offline   Reply With Quote
Old 01-07-2018, 16:23   #5
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 5,228
Re: Chart File Compression

Quote:
Originally Posted by yachtvalhalla View Post
Excuse my ignorance but the above means nothing to me except I think you are saying there is a limit to files greater than 2.048GB?


The folder that fails for me is 3.44GB in size.



Also, I was able to compress the full cm93 file from 1.44GB to 910MB.


Terry
There is no limit, what did-g says is that the final reported numbers may get wrong if the total size is bigger than 2048MiB.

@did-g - From briefly looking at the code, all the vars involved seem to be unsigned long so as far as I can tell should be OK...
What seems to be a possible problem when compressing a lot of charts is that I don't see the loop ever yielding to let the main event queue to be processed which we know to be problematic on some platforms...
nohal is offline   Reply With Quote
Old 01-07-2018, 16:52   #6
Registered User

Join Date: Jun 2015
Posts: 370
Re: Chart File Compression

Quote:
Originally Posted by nohal View Post
There is no limit, what did-g says is that the final reported numbers may get wrong if the total size is bigger than 2048MiB.

@did-g - From briefly looking at the code, all the vars involved seem to be unsigned long so as far as I can tell should be OK...
What seems to be a possible problem when compressing a lot of charts is that I don't see the loop ever yielding to let the main event queue to be processed which we know to be problematic on some platforms...
Hi,

Ah OK,unsigned long is 32bits on linux so if uncompressed size is > 4GB it's rollover and you can get uncompress size < compressed, anyway it's cosmetic.


Shouldn't dialog box Update call yield? I'll try to test on win 10 later.
BTW how much work would be needed for a window 64 version?

Is an installer with both versions possible?
did-g is offline   Reply With Quote
Old 01-07-2018, 17:43   #7
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 13,936
Re: Chart File Compression

Are you guys compressing the SENC files and then using them?
rgleason is offline   Reply With Quote
Old 01-07-2018, 17:48   #8
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 5,228
Re: Chart File Compression

Quote:
Originally Posted by did-g View Post
BTW how much work would be needed for a window 64 version?
All the dependencies would have to be built (and maintained) - very high on my annoyance-meter already now with 32bits only... For OpenCPN itself not that much work. This would get "better" switching to libs from vcpkg where possible and dropping support for XP.
Quote:
Is an installer with both versions possible?
Yes, but the installer itself of course must be 32bit. Development of NSIS related stuff also not exactly what I enjoy doing.
nohal is offline   Reply With Quote
Old 01-07-2018, 17:50   #9
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 5,228
Re: Chart File Compression

Quote:
Originally Posted by rgleason View Post
Are you guys compressing the SENC files and then using them?
No. We are compressing the charts by clicking the "Compress Selected" button on the Chart files tab.
nohal is offline   Reply With Quote
Old 02-07-2018, 04:33   #10
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 13,936
Re: Chart File Compression

Just to be clear, this is about the Options > Chart > Chart Files


"Prepare all ENC" button?
rgleason is offline   Reply With Quote
Old 02-07-2018, 04:35   #11
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 5,228
Re: Chart File Compression

Quote:
Originally Posted by rgleason View Post
Just to be clear, this is about the Options > Chart > Chart Files


"Prepare all ENC" button?
No, this is about the "Compress Selected" button.
nohal is offline   Reply With Quote
Old 02-07-2018, 17:01   #12
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 13,936
Re: Chart File Compression

What version and OS please?
rgleason is offline   Reply With Quote
Old 02-07-2018, 17:49   #13
Registered User
 
yachtvalhalla's Avatar

Join Date: Aug 2009
Location: Philippines
Boat: Fuji 32 Ketch
Posts: 849
Re: Chart File Compression

Quote:
Originally Posted by rgleason View Post
What version and OS please?

I am using 4.8.4 and W10.


I have a KAP chart folder of 3.42GB with 598 files in 63 folders. Ran the compression for over 5 hours. All of the files have a .xz suffix added to the .KAP suffix but the folder size is still 3.42GB.


Smaller folders seem to get reduced in size. For example an 885MB folder was compressed to 426MB and a cm93 (1.4GB) compressed to 866MB.


Terry
__________________
VALHALLA's Mooring Page
yachtvalhalla is offline   Reply With Quote
Old 03-07-2018, 03:21   #14
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 13,936
Re: Chart File Compression

I must be blind. "Compress Selected" is on the upper right under "Remove selected"! So dumb. I have never used this. Does is slow down panning and zooming?
rgleason is offline   Reply With Quote
Old 03-07-2018, 03:35   #15
Registered User

Join Date: Jun 2015
Posts: 370
Re: Chart File Compression

Quote:
Originally Posted by rgleason View Post
I must be blind. "Compress Selected" is on the upper right under "Remove selected"! So dumb. I have never used this. Does is slow down panning and zooming?
Hard to tell on a PC zooming is likely slower, a little, but if you're using a slow flash memory it could be faster, any with texture cache KAP aren't used that much.
__________________

did-g is offline   Reply With Quote
Reply

Tags
compression

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How to Convert "*.route" Route File into ".*wpt" File To Be Used in Maxsea V.10 doiroda Navigation 7 12-03-2020 06:00
chart File won't upload osprey877 OpenCPN 10 15-10-2016 19:35
Chart with file Extension .CAP display problems garry_johnson44 OpenCPN 3 18-08-2015 17:54
Open CPN - Chart 'Mark' file copying livedelife Navigation 7 15-02-2013 16:07
3.0.0 : make package : bad file right after installing the deb file manuprenlair OpenCPN 1 12-06-2012 04:17

Advertise Here


All times are GMT -7. The time now is 02:39.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2020, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.