Import GPX Tracks Crash

[dacust]

Originally Posted by j ferguson

Tore,
for the ICW, 60 foot intervals are useful. It might be that using time intervals instead of distance to create the tracks would make more sense because i would get higher resolution when going slower, but then the difference between 6 knots and 2 or 3 is not that great.

i have no problem with these large track files with the software i now use, which also runs on Linux. I have all 10 years of tracks available on a tab (twenty - most of which are more than 6 megs) and i can easily display 2 or 3 while recording the "current" track.

From discussions with other software developers, my use of tracks is not a high-demand feature.

OpenCPN was highly recommended by a friend whose opinion i respect, but he couldn't understand why i wanted these big tracks either. Ah well..

john

FWIW, I have the same kind of use as you for long tracks.

The first time I posted in here that I was having performance problems once my route got over 1000 waypoints, people thought I was crazy. But, with a combination of ICW and rivers, planning out a 2000 mile trip can easily have 3000+ waypoints.

Having all those waypoints sounds crazy until the first time you wait at a lock until after dark, then have to thread some islands in the dark to find an anchorage. Now, the primary things being used are visual, but the waypoints are invaluable to help you know the general direction.. I've had that insane level of detail help me out at least once each long trip I've made.

-dan

[sinbad7]

John..

You can easily use ANY of your imported .gpx tracks in OCPN as long as you reduce the frequency of your waypoints.

Using the FREE RouteConverter,I have in the attached example reduced a track file with all of 15.817 waypoints sampled every second,down to a mere 82 sampled every minute.

As you see from the Delete positions box,it is fairly configurable to your own special requirements.

Tore

[Bluesipp]

Originally Posted by bdbcat

Folks....

I've studied one of the large GPX files question.

Conclusion:
OpenCPN 2.5 does some things well, some not so well.

One thing we do not do well is this:

Single tracks of 1000 miles with trackpoints every 60 ft. (i.e. about 100,000 points in a single track) are not well supported by OpenCPN.
They work OK, but load too slowly to be usable.

It's a design decision driven by portability and conformance to industry standards. We store extended XML information for all trackpoints and waypoints in our navobj.xml files using a common format. The processing cost eventually becomes unreasonable for the resulting XML files when tracks of this size are imported.

If this is a general problem for enough users, then it may be posted as a FlySpray bug for further consideration in future versions.

Sorry, but there it is.

Thanks
Dave

Dave

Very many thanks for your message.
I seem to get problems when the GPX file size approaches 1.5 MB. Which I think is smaller than you suggest. But these files are not native OpenCPN files, they are converted from PTF files and perhaps they carry additional data that OpenCPN does not easily handle. Just a thought and I don't claim to understand any of the technical details!

Cheers
John (s/y Bluesipp)

[j ferguson]

Originally Posted by bdbcat

Folks....

I've studied one of the large GPX files question.

Conclusion:
OpenCPN 2.5 does some things well, some not so well.

One thing we do not do well is this:

Single tracks of 1000 miles with trackpoints every 60 ft. (i.e. about 100,000 points in a single track) are not well supported by OpenCPN.
They work OK, but load too slowly to be usable.

It's a design decision driven by portability and conformance to industry standards. We store extended XML information for all trackpoints and waypoints in our navobj.xml files using a common format. The processing cost eventually becomes unreasonable for the resulting XML files when tracks of this size are imported.

If this is a general problem for enough users, then it may be posted as a FlySpray bug for further consideration in future versions.

Sorry, but there it is.

Thanks
Dave

Hi Dave,
After a couple of weeks of thinking about the navobj.xml file, my question is why would you store formatted data, especially when the formatting seems so burdensome? If one runs tracks and collects routes, eventually this file could become a a few megs. To then require that it be loaded in order to fire up OpenCPN and necessarily slowly because of having to read all the formatting seems counterintuitive.

I can imagine someone fooling with navobj.xml manually, but since it appears to be spun off from cached data, why not store the cache, then on next startup, quickly look at navobj.xml to see if it's the same as what was cached during the last session and and if it's intact, then run with the binary stuff?

I may have missed the whole point of this, but portabiity, open source, and standards don't seem to me to justify taking 70 to 80 megs to store data that can be confined to 5 or 6 megs (or less, much less if binary) without the formatting.

Help me understand what I'm missing here.

john

[andyalford]

Originally Posted by Bluesipp

Dave

... But these files are not native OpenCPN files, they are converted from PTF files and perhaps they carry additional data that OpenCPN does not easily handle. ...

Cheers
John (s/y Bluesipp)

I have some PTF files I would like to import to OpenCPN - please can you advise how you achieved this? (Thankfully they are quite small!)

Cheers, Andy

[Jon Hacking]

It would be very nice if the redundant points from a straight track could be easily removed from a track file. It would also be nice if OCPN could edit the tracks to remove those redundant points.

These intermediate points in a straight track form the majority of track-file bloat, but I don't have a good algorithmic way to remove them. So I import the tracks into MaxSea, edit them there to remove the redundant intermediate points, & then convert them back to GPX files for posting on our Track Files Download page.

Removing the bloat improves both download times & OCPN performance when loaded. Our Australia & Indonesia tracks are both well over 1,000nm, & are both only ~35KB.

Andy, Welcome to Cruisers Forum! The best way I've found to convert between OCPN (gpx) & MaxSea (ptf) files is a UK program called GPS Utility. There's a free trial version that's severely limited in file size, but the full version costs ~$60. We use it all the time, but we also have lots of track files from many cruisers that we publish (in both gpx & ptf formats) for downloading.

One cool feature of GPSU is that it produces Google Maps files, so I can import the tracks directly into a Google Map & embed that map on our website so users can see exactly what they're downloading. But GM is also severely limited by file size, so this is another reason I'd love a better way to remove track-file bloat.

[PjotrC]

Originally Posted by Jon Hacking

It would be very nice if the redundant points from a straight track could be easily removed from a track file. It would also be nice if OCPN could edit the tracks to remove those redundant points.

I posted a simple trial solution to a similar problem Forums - Post 667498">here. If there is enough interest in this, it might be revived and improved.

[Jon Hacking]

Originally Posted by PjotrC

I posted a simple trial solution to a similar problem Forums - Post 667498">here. If there is enough interest in this, it might be revived and improved.

Piotr, Thanks for this! I am very interested - Anything I can do to help it along?

[j ferguson]

Guys, it isn't just the redundant points in a straight course, it is the unbelievable need to have a formatted navobj.xml file which takes 6 megs of track points and expands them to 70 megs without adding an iota of value. I asked above, why this was a good idea - dead silence.

[bdbcat]

j ferguson....

Patience, please. Its on the list.

Thanks
Dave

[PjotrC]

I am not happy with the size increase of .gpx files.

But the size in Bytes is not directly related to import speed. Here are some timings for an 18000 points track:

tk18k           - 11489789 B        5 s        15 s
tk18k_noblank   -  7590280 B        5 s        14 s
tk18k_noguid    -  6309364 B        5 s        13 s
tk18k_nocpn     -  4953141 B        5 s        13 s        

import time sec                    2.5.0       2.5.0 
                                   2 core      1 core

tk18k is as exported
tk18k_noblank has multiple blanks suppressed
tk18k_noguid has "guid" lines removed as well
tk18k_nocpn has all "opencpn:viz" lines removed as well

I still hope for a different format for save/restore of navobj.xml and import/export for .gpx.

But it would be a sad day when I would need to use a Hex Editor instead of Notepad to look at OCPN data.

Piotr