How to eliminate Names and other Waypoint attributes in GPX files

[derfy]

I use GPX route files all the time, moving them from Navionics into OpenCPN and onto my Raymarine Axiom (to drive the boat).

I find the OpenCPN exported GPX files are full of clutter for each waypoint (technically these are route points). Attributes like <time>,<name>,<sym>,<type>,<extensions>, making the files much larger than need be.

Worse, the silly Raymarine chartplotter dutifully shows me each and every route point name on the screen - cluster I don't need.

So I manually edit out all this stuff, but it is too big a job.

Q: Can I suppress the export of all these extra attributes and just get route point lat and lon?

Q: Is there a good editor or script that can do this. I have tried some xml editors but they did not work out. Maybe there is a good one? Or just a nice script someone has written?

[nohal]

No, you can't suppress the information that OpenCPN exports. All that data is actually useful and compliant with the GPX standard, so should not be any problem for a well educated consumer.

There are several ways to manipulate GPX filesusing Python described at https://stackoverflow.com/questions/...-beautifulsoup

Another approach using the xmlstarlet toolkit is described at https://oleb.net/2020/sanitizing-gpx/ - This one is probably very close to what you are trying to achieve.

[Antipole]

So you could go the way Nohal has suggested.

There is another way - you could use JavaScript to write a bespoke .gpx file containing just the bits of a route you want. That would be a straightforward and interesting project.
I have published a script that writes my tracks in a bespoke way into a .kml file. You could take that as a starting point and modify it to what you want. Like .gpx, .kml is an .xml file so the technique for writing the data out can be the same.

My script constructs a single track .kml file from a sequence of OpenCPN tracks. That is much more complicated than you need so you could cut much of it. You would need to read in the routes you want - suggestion: those that are visible - and then write the required bits out as .gpx.

In my script, the data is written by a series of calls to the function out()
Set variable log to true and it will 'print' the output in the display pane so you can see what is happening.

See here for my published script.

[derfy]

Originally Posted by Antipole

So you could go the way Nohal has suggested.

There is another way - you could use JavaScript to write a bespoke .gpx file containing just the bits of a route you want. That would be a straightforward and interesting project.
I have published a script that writes my tracks in a bespoke way into a .kml file. You could take that as a starting point and modify it to what you want. Like .gpx, .kml is an .xml file so the technique for writing the data out can be the same.

My script constructs a single track .kml file from a sequence of OpenCPN tracks. That is much more complicated than you need so you could cut much of it. You would need to read in the routes you want - suggestion: those that are visible - and then write the required bits out as .gpx.

In my script, the data is written by a series of calls to the function out()
Set variable log to true and it will 'print' the output in the display pane so you can see what is happening.

See here for my published script.

Thanks. A generous offer. But it has been too many years since I wrote any code. I could barely remember the linux BASH commands to boot up and copy some files. I was just hoping somebody else had done this and would say "here ya go" (on Win 11 please). I have maybe 100 gpx files I would like to scrub in this manner. It is a tedious job, and one that is not without peril. One mistake and you blow away a route point that might be critical.

[derfy]

Originally Posted by nohal

No, you can't suppress the information that OpenCPN exports. All that data is actually useful and compliant with the GPX standard, so should not be any problem for a well educated consumer.

There are several ways to manipulate GPX filesusing Python described at https://stackoverflow.com/questions/...-beautifulsoup

Another approach using the xmlstarlet toolkit is described at https://oleb.net/2020/sanitizing-gpx/ - This one is probably very close to what you are trying to achieve.

Thanks, I will investigate the Python reference. Could be the thing I need.

Sounds like you think my request is unreasonable. I do not assert that the OCPN exports are non-compliant or not useful.

But, for me, all this data is not useful. The info I need to see on my chart plotter when piloting is just a bright line between two points, and whether the boat is to the right of left of the line.

Piloting is very different than route planning where, yes, it is nice to have annotations, etc. When piloting, AIS/Radar targets and depth contours are much more important to see than machine-generated intermediate route point names.

[nohal]

Originally Posted by derfy

Thanks, I will investigate the Python reference. Could be the thing I need.

Sounds like you think my request is unreasonable. I do not assert that the OCPN exports are non-compliant or not useful.

But, for me, all this data is not useful. The info I need to see on my chart plotter when piloting is just a bright line between two points, and whether the boat is to the right of left of the line.

Piloting is very different than route planning where, yes, it is nice to have annotations, etc. When piloting, AIS/Radar targets and depth contours are much more important to see than machine-generated intermediate route point names.

The second approach using xmlstarlet will likely be easier for you to implement if what you really want to do is what you describe (just removing some of the data in the GPX). Using Python is pretty much the same in complexity as using Javascript, which you say you don't want to.

[CaptTom]

Originally Posted by nohal

No, you can't suppress the information that OpenCPN exports. All that data is actually useful and compliant with the GPX standard, so should not be any problem for a well educated consumer.

The OP made a very reasonable request. This is a common need.

One of the first things I did when I started using OpenCPN years ago was write a macro in my old text editor to strip out the unwanted lines. I still use it, outdated though the software may be, because it works for me. I "clean" every single GPX file I export from OpenCPN.

I can't agree that all the data are useful. Just because they are syntactically legal in the GPX standard doesn't mean everyone wants or needs them. All the different methods people use to strip those lines out, some of which are mentioned and linked here in this thread, are proof enough that this would be a valuable feature to add to OpenCPN.

[nohal]

The data in the GPX files we export are there because they are useful, if they wouldn't be there, information would be lost.
The consumer of the data is responsible for discrading what the consumer is considering not useful for him in the GPX workflow. The producer is responsible for producing data conformant with the GPX standard, which we do.
If the consumer you are using can't handle the data available in a way that suits you, you simply help it by preprocessing the data using the many methods available, some of which are linked in this thread.

[derfy]

Originally Posted by nohal

The data in the GPX files we export are there because they are useful, if they wouldn't be there, information would be lost.
The consumer of the data is responsible for discrading what the consumer is considering not useful for him in the GPX workflow. The producer is responsible for producing data conformant with the GPX standard, which we do.
If the consumer you are using can't handle the data available in a way that suits you, you simply help it by preprocessing the data using the many methods available, some of which are linked in this thread.

I am grateful for all the data I get from OCPN - it is a fantastic tool provided by really selfless volunteers. And, it is certainly no one's responsibility to implement my request. It's just a request. And it is clear that if I am to ever get scrubbed xml files with only lat and long, it seems it will be my "responsibility" to do it. I am glad we have the responsibilities of everyone clarified, thank you.

I would just rather be sailing and was hoping someone had done this already.

I am curious why eliminating optional xml fields and extensions would make the exported xml not "compliant"? I am left to wonder what exacting standard demands the presence of these fields? Navionics gpx exports, for example, do not include machine-generated route point names. Maybe they are "non-compliant" but their outputs are a lot more useful in my chart plotter.

Sorry I am not swift enough to bring up a dev environment, gather all the libraries and tools, learn the programing language of the day, and debug the contrib source code to do this simple thing. Luckily, I am the only one who seems to need it, so it is no problem for anyone else.

[nohal]

Nobody ever said that a GPX file is not compliant with the standard if it does contain rtept elements without names.
A GPX file is compliant with the standard as long as it conforms to the schema defined by the standard published at https://www.topografix.com/gpx.asp, simple as that.

Of course a route may contain points with other than machine generated names, regardless of originating from OpenCPN or other producer.

I am sorry that your chartplotter does display route point names by default and does not offer you a way to change that behavior, but it is simply not the responsibility of the side producing the original GPX to solve this. You already have received pointers to reasonably simple ways to achieve it in your workflow.

[derfy]

Originally Posted by nohal

Nobody ever said that a GPX file is not compliant with the standard if it does contain rtept elements without names.
A GPX file is compliant with the standard as long as it conforms to the schema defined by the standard published at https://www.topografix.com/gpx.asp, simple as that.

Of course a route may contain points with other than machine generated names, regardless of originating from OpenCPN or other producer.

I am sorry that your chartplotter does display route point names by default and does not offer you a way to change that behavior, but it is simply not the responsibility of the side producing the original GPX to solve this. You already have received pointers to reasonably simple ways to achieve it in your workflow.

I would be very happy to learn of a way to suppress the visibility of the route point names on my new Axiom Pro S Chartplotter. That would solve the problem just was well. I cannot find a way. I had the problem with my older Raymarine C-Series that it did not like duplicate route point names, even when the route points are contained in different routes. I just had to manually edit the names out of the files.

I also tried just opening the gpx files in Excel as an xml file, and just delete the columns I did not want. But Excel won't save the edited file, giving me some errors related to not having a schema. Couldn't get around it.

Oh well, sailing in the digital world is so much fun. I should go back to paper charts.

[CaptTom]

The issue seems to be that there's a lot of useless information in the GPX file. It doesn't seem unreasonable to suggest it would be a nice feature to be able to limit this.

Maybe an example would help. Here's the beginning of a route I just exported from OpenCPN:

  <rte>
    <name>01 Hudson River to Waterford</name>
    <extensions>
      <opencpn:guid>55c9ffff-e911-45f3-9940-c32b565effff</opencpn:guid>
      <opencpn:viz>1</opencpn:viz>
      <opencpn:planned_speed>6.00</opencpn:planned_speed>
      <opencpn:time_display>UTC</opencpn:time_display>
      <gpxx:RouteExtension>
        <gpxx:IsAutoNamed>false</gpxx:IsAutoNamed>
      </gpxx:RouteExtension>
    </extensions>
    <rtept lat="40.697196895" lon="-74.022932398">
      <time>2022-09-19T18:12:13Z</time>
      <name>NY Battery</name>
      <desc>NY Harbor S of The Battery</desc>
      <sym>square</sym>
      <type>WPT</type>
      <extensions>
        <opencpn:guid>2a370000-8705-4da7-8c4e-e9455f3fffff</opencpn:guid>
        <opencpn:arrival_radius>0.050</opencpn:arrival_radius>
        <opencpn:waypoint_range_rings visible="false" number="0" step="-1" units="-1" colour="#FFFFFF" />
        <opencpn:scale_min_max UseScale="false" ScaleMin="2147483646" ScaleMax="0" />
      </extensions>
    </rtept>

By my count, 17 of these 24 lines are useless. None of the extensions will be recognized by any other software or firmware I use. The symbol will probably also not be used. The time is basically arbitrary, based on when I created the route and has no bearing on any reality.

I have hundreds of these files. Something as simple as a check box to disable extensions would cut the storage requirements and shorten the transmission time considerably, with no loss of utility for most users. Of course, since it would be an option, users would still be able to choose to keep all that data, for example to send to another OpenCPN user.

If the OP and I were the only two who wanted this feature, dismissing us as irrelevant would be appropriate. But from the number of ways people have designed to overcome this limitation, I surmise that it would be well received by the user community.

[nohal]

By my count every single line removed means lost information. If some of it is useless in your particular usecase, remove it at will.

[Capt.Don]

This is a topic I've pondered for a while. Notepad++, my favorite Windows text editor, supports search and replace using regular expressions. A brief internet search returns a regex search condition that replaces text between tags. Note, this doesn't support nested tags.

In my archived opencpn and navionics gpx route files, I wanted to remove <ele>...</ele> and all the <extensions>...</extensions>.

Here's the search regex to remove the <ele> tags,
<ele>[^<>]+</ele>

Because the <extensions> have nested tags, I used a similar regex to remove the embedded tags, then the extensions.

For navionics gpx files,
<navionics_speed>[^<>]+</navionics_speed>
<navionics_haccuracy>[^<>]+</navionics_haccuracy>
<navionics_vaccuracy>[^<>]+</navionics_vaccuracy>
<extensions>[^<>]+</extensions>

For opencpn gpx files
<opencpn:guid>[^<>]+</opencpn:guid>
<opencpn:viz_name>[^<>]+</opencpn:viz_name>
<opencpn:arrival_radius>[^<>]+</opencpn:arrival_radius>
<opencpn:waypoint_range_rings[^<>]+/>
<opencpn:scale_min_max[^<>]+/>
<extensions>[^<>]+</extensions>

For now, I'm only removing the <ele> tags and will think about the rest.

Don

[wholybee]

Check out GPS Babel. It will convert and filter GPX files. It is very powerful and much easier than manually editing anything.

https://www.gpsbabel.org/

I do agree that when exporting a GPX file, there should be a dialog with some check boxes asking what to include. Suggesting it is up to the "consumer" is very unfriendly. That leads to complicated workflows, incompatible software, and time sucks for the "consumer." I would also like to suggest a similar dialog for GPX import. For example, when I import a route from Predictwind, it plots not only the route, but all the waypoint icons and names each of them on the screen. It is basically unusable without a large amount of work (or using some other program to convert it)

A while back there was a thread on OpenCPN interoperability and what changes/features should be there. I'm not sure this made it on that thread, but it really needs to be, and in the scheme of things, features to add, etc. shouldn't be that hard. Please do not dismiss it so casually.