Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Rate Thread Display Modes
Old 03-09-2010, 14:45   #1
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,247
OpenCPN.org and the Pilot Charts

As you all know, South Bay Network kindly and generously, donates opencpn.org web space and bandwidth.
The success of the new feature - Pilot Charts as OpenCPN charts - may just strain this, much appreciated, hospitality a bit.
The fact is that since the Pilot Charts were published, the bandwidth used by the OpenCPN project is more than three times greater than previously.
Another way to see it, is that we now use about 50% more than the limit that SBN is comfortable with.
This gives the the OpenCPN project some hard choices. It boils down to this.
* We can withdraw the download feature of the Pilot Charts from the opencpn.org site, and trust that users can find other download facilities.
The CNF.SeaSoft site already has the Pilot Charts available for download
* We can try to find other ways to host this resource hog, the files are huge. This is where we need ideas and suggestions from all OpenCPN users.
* A third, in my mind not very attractive option, is to somehow finance the OpenCPN.org with advertising.

All the web statistics for opencp.org is available here https://opencpn.org/plesk-stat/webstat/

So.... are the Pilot Charts worth keeping ......??
__________________

__________________
cagney is offline   Reply With Quote
Old 03-09-2010, 15:08   #2
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 4,016
Thomas,
what about some kind of a mirror system where the expenses could be shared? The code needed to select a mirror will be couple of lines. I'm ok with providing some bandwidth from my servers...
Another option is hosting the files on sourceforge.net - I have seen bigger downloads there.

Pavel
__________________

__________________
nohal is offline   Reply With Quote
Old 04-09-2010, 08:40   #3
Senior Cruiser
 
idpnd's Avatar

Cruisers Forum Supporter

Join Date: Sep 2007
Location: Almerķa, ES
Boat: Chiquita 46 - Libertalia
Posts: 1,551
Is it possible to host on sourceforge? whats the pilot chart license vs sf.net license requirements?
__________________
sv Libertalia
idpnd is offline   Reply With Quote
Old 04-09-2010, 11:40   #4
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,247
I really had to do some quick changes to the download links, that will work for now.
Sourceforge is an option, but then really only if we start a new project, "Open source chart conversion tools", or something like that. I'm not to keen on this alternative.
Dave has good reasons for not wanting to use the OpenCPN project for these kind of files.
It will be interesting to see what kind of user reactions we will get, if any.... as a result of this change.

Thomas
__________________
cagney is offline   Reply With Quote
Old 04-09-2010, 11:48   #5
Registered User

Join Date: Dec 2009
Location: Ontario
Boat: 5 m Bolger runabout, plus a Starwind 860 power tri under construction
Posts: 248
Quote:
We can try to find other ways to host this resource hog, the files are huge. This is where we need ideas and suggestions from all OpenCPN users.
May I suggest offering the charts as a .torrent instead of a direct download? Host one seed copy on the server (throttling the bandwidth to whatever your ISP wants), and rely on the users to seed additional copies when they're on connections that can handle it.

The fact that media pirates like to use the Bittorrent protocol doesn't change the fact that it's a reliable, robust system designed for precisely the kind of situation OpenCPN is facing with these chart downloads (large data files, distributed to many users, by an organization with no money or bandwidth of its own).

I definitely want to see the charts kept... they can be quite hard to find and mixed with malware links at times. If OpenCPN can't host them itself, let all the users who want them share a bit of their own bandwidth to keep them available.

DepositFiles works, but is insanely annoying. I'd be happy to create the torrents for these charts if it's deemed desirable. It will take a few days to get them off DepositFiles before I can do so....
__________________
Matt Marsh
marshmat is offline   Reply With Quote
Old 06-09-2010, 17:54   #6
Registered User

Join Date: Dec 2009
Location: Ontario
Boat: 5 m Bolger runabout, plus a Starwind 860 power tri under construction
Posts: 248
OK, let's give this a try.

Here's the torrent file for the pilot charts:
http://www.marsh-design.com/sites/de...N_2010.torrent

The files included are:
PilotCharts_IO.zip
PilotCharts_NAC.zip
PilotCharts_NAN.zip
PilotCharts_NAW.zip
PilotCharts_PDF_IO.zip
PilotCharts_PDF_NA.zip
PilotCharts_PDF_SP.zip
PilotCharts_SP_Charts.zip
PilotCharts_SP_Info.zip
The complete set is 583 MB. The BSB charts alone are 344 MB if you omit the PDF sets (which can be obtained, one page at a time, from the NGA website).

I'm seeding the thing now. You'll need a Bittorrent client to open the torrent file and begin the download. It will be pretty slow for the first few people, until a "swarm" of complete seed copies can form.

If it works, you'll need to leave the torrent open for a few days after you finish the download, so that your computer can help seed it to everyone else.

For those unfamiliar with Bittorrent: More about how it works here on Wikipedia... or just download a BT client like Vuze or uTorrent, click the link above to open the .torrent file, and wait.
__________________
Matt Marsh
marshmat is offline   Reply With Quote
Old 06-09-2010, 19:26   #7
Obsfucator, Second Class
 
dacust's Avatar

Cruisers Forum Supporter

Join Date: Feb 2008
Location: Southeast USA.
Boat: 1982 Sea Ray SRV360
Posts: 1,743
Quote:
Originally Posted by marshmat View Post
I'm seeding the thing now. You'll need a Bittorrent client to open the torrent file and begin the download. It will be pretty slow for the first few people, until a "swarm" of complete seed copies can form.
I am downloading it. Says 2 hours 13 minutes to go.

I can leave it active indefinitely, so it should help other in the future.

-dan
dacust is offline   Reply With Quote
Old 06-09-2010, 21:38   #8
Obsfucator, Second Class
 
dacust's Avatar

Cruisers Forum Supporter

Join Date: Feb 2008
Location: Southeast USA.
Boat: 1982 Sea Ray SRV360
Posts: 1,743
Seeding, now. But no takers, so far.

-dan
dacust is offline   Reply With Quote
Old 06-09-2010, 23:42   #9
Registered User

Join Date: Aug 2009
Posts: 15
Hi,

In the stats does that traffic amount include the downloading of the chart files, as 100GB a month doesn't seem that high

If it does include the chart downloads, I can host the download files until it goes over 200gb per month
__________________
scubataz is offline   Reply With Quote
Old 07-09-2010, 07:24   #10
Registered User

Join Date: Dec 2009
Location: Ontario
Boat: 5 m Bolger runabout, plus a Starwind 860 power tri under construction
Posts: 248
I think the August stats include the charts- hence the jump from ~20 to ~100 GB/month. That's not much by commercial standards, but a big deal to a FOSS project using donated server resources.
If you could host a seed of the torrent, scubataz, that'd be great
__________________
Matt Marsh
marshmat is offline   Reply With Quote
Old 07-09-2010, 08:09   #11
Registered User

Join Date: Aug 2009
Posts: 15
VPS and torrents do not go well together, hits the io too much in the shared environment, best I can offer is a full file download spot for you with ftp access and http access for people to download
__________________
scubataz is offline   Reply With Quote
Old 07-09-2010, 09:46   #12
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,247
Thanks marshmat for setting up the torrents. I'm downloading now (ktorrent) and will seed later on.
scubataz, thank you very much for the offer, seems great to me.
I would like to link to the torrent and really to as many download sites as possible on the opencpn.org download side , that are not
Quote:
insanely annoying.
to the users.
Is that Ok with your torrent Matt?
scubataz, send me the details, I'll add a link. Can you throttle the bandwidth yourself?

Ooops, very quick download, already seeding

Thomas
__________________
cagney is offline   Reply With Quote
Old 07-09-2010, 10:48   #13
Registered User

Join Date: Aug 2009
Posts: 15
I will send a pm with the ftp download details, do you want to setup a subdomain pointer to the download site eg: mirror-01.opencp.org etc?

I can throttle bandwidth if need be, will keep an eye on it and see how it goes
__________________
scubataz is offline   Reply With Quote
Old 07-09-2010, 13:12   #14
Senior Cruiser

Cruisers Forum Supporter

Join Date: Aug 2009
Location: between the devil and the deep blue sea
Boat: a sailing boat
Posts: 17,314
filehippo? brothersoft? softpedia?

private hosting?

Why cannot the Pilot Charts files be had right from the source (NOAA?)

barnie
__________________
barnakiel is offline   Reply With Quote
Old 07-09-2010, 13:27   #15
Registered User

Join Date: Dec 2009
Location: Ontario
Boat: 5 m Bolger runabout, plus a Starwind 860 power tri under construction
Posts: 248
Re. torrents: Just put a copy of the .torrent file (about 24 kB) on whatever Web page you want. The actual data transfer is entirely peer-to-peer and they become self-sustaining with no server involvement once a handful of people complete the download and start seeding.

Some people may prefer a direct download (thanks scubataz for offering to host one)- perhaps best to link to this, but advise that it's a secondary choice so as not to overload scubataz's server too?

I hope we can offer more charts this way in the future.

Barnie: The charts can be had directly from the NGA as free downloads, but in formats that can't be integrated into OpenCPN. The ones in the torrent (see post 6) were repackaged by the OpenCPN team into a more useful format (BSB raster).
__________________

__________________
Matt Marsh
marshmat is offline   Reply With Quote
Reply

Tags
charts

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
OpenCPN.org - Feature Requests Netsurfer OpenCPN 8 14-05-2017 01:15
Pilot Charts Now Available as Digital Charts cagney OpenCPN 21 11-09-2010 14:29
OpenCPN.org DNS Problem? zemaria OpenCPN 17 29-03-2010 07:36
OpenCPN.org Announcement manimaul OpenCPN 31 10-03-2010 15:25



Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 16:49.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.