How to eliminate Names and other Waypoint attributes in GPX files

[derfy]

Ok. I was looking for just one good example (other than lat and lon) of useful data for piloting (not auto-piloting BTW). What I read in your reply is lengthy discourse pointing out....

a) that exporting a "route" is not really a valid thing to expect. gpx data is just data (do you really believe that, btw?)

b) the function of a schema is important, and I should care

c) the exported data is compliant with standards

d) that complete and rich of data is always superior for application interoperability, and withholding data is considered rude (if not incompliant)

e) it the job and role of receiving apps to appropriately filter extraneous data, and that this is easy since the internal OCPN info is clearly marked.

f) OCPN does not and ought not care what kind of apps are using its data (you mention auto-pilots, I have not)

Then finally you make a case that an autopilot app might care about when a route point was generated (by your app), and the name (that your app) assigned to it. Or maybe the symbol you used in your app. I missed the convincing "why" it should actually care, but, yes, I suppose it might.

Finally, in case I missed the point, you summarize that this exported data is really just a rich collection of possibility important or useful information that the user is free and obligated browse and choose at will.

And, once more, I should remember that it is fundamentally and irrevocability the role of the importing app to choose its appropriate data.

Did I get something wrong? Have missed your point of view? If I got it wrong, please hammer it home a few more times.

My only reply ...

Verbosity Obfuscates.


Your torrent of verbiage has carried the day. I surrender. And, I am very glad Navionics -unlike OCPN - was designed for use by people who drive boats.

[derfy]

Originally Posted by Portage_bay

Derfy,

Getting back to your original question and avoiding the ensuing discusson I can offer a solution to strip out the unwanted lines in a standard .GPX file. This solution assumes you are transfering GPX files to your Raymarine system. I will admit upfront to being ignorant of Raymarine. This is a variation on a process I have used for many years.

1. Create your route in any app you wish, in your case Navionics.
2. Export the route file to storage media which depends upon the hardware you are using.
3. Open the GPX file in GPS Utility
4. Edit anything if you wish
5. Export to the file format you require
6. Send it off to your Raymarine device

...

Thanks, I will try it.

BTW Navionics gpx exports are just fine. No route point names clutter my Ray Axiom screen.

The issue is OCPN. It generates route point names that are meaningless and these are visible on the Axiom screen - hiding the AIS traffic I really need to see. OCPN also sends huge files packed with useless internal info elements.

Here is what Navionics generates (pretty simple and no problem)....
<?xml version="1.0" encoding="UTF-8" ?><gpx xmlns="http://www.topografix.com/GPX/1/1" version="1.1" creator="Navionics Boating App"><metadata><link href="http://www.navionics.com" /></metadata><rte><name>JAM2NYHS</name><rtept lat="40.567862" lon="-73.911066" /><rtept lat="40.565628" lon="-73.918800" /><rtept lat="40.562470" lon="-73.930819" /><rtept lat="40.557585" lon="-73.940898" /><rtept lat="40.554324" lon="-73.944895" /><rtept lat="40.549349" lon="-73.945946" /><rtept lat="40.540503" lon="-73.943377" /><rtept lat="40.530921" lon="-73.944976" /><rtept lat="40.527110" lon="-73.945174" /><rtept lat="40.513290" lon="-73.928555" /><rtept lat="40.485401" lon="-73.906799" /><rtept lat="40.428393" lon="-73.906700" /></rte>

[nohal]

And isn't the issue your Axiom display showing point names over the AIS targets? Or not allowing you to display the data you have imported to it the way you want? This discussion became pointless and you received indications on multiple solutions to tailor your workflow to do exactly what you want, so we can probably close it.

[derfy]

Originally Posted by nohal

And isn't the issue your Axiom display showing point names over the AIS targets? Or not allowing you to display the data you have imported to it the way you want? This discussion became pointless and you received indications on multiple solutions to tailor your workflow to do exactly what you want, so we can probably close it.

Yes, you have the issue. It is clearly Raymarine's deficiency. I will campaign vigorously with them to change their product.
It does seem pointless to suggest something to the OCPN crowd. I agree with that. Ciao.

[nohal]

Well, you have absolutely no problem campaigning vigorously here...

[Be Free]

Originally Posted by derfy

Yes, you have the issue. It is clearly Raymarine's deficiency. I will campaign vigorously with them to change their product.
It does seem pointless to suggest something to the OCPN crowd. I agree with that. Ciao.

I'm sorry but you have missed the most important point. Everything, your Raymarine, your Navionics, and OpenCPN are all working exactly as they should. In both cases the exporting programs are exporting what they want to export and the importing program is importing what it understands.

I don't believe that it is Raymarine's deficiency. As I pointed out several posts ago, your Raymarine has no issue with the data. It has ignored everything that it does not want or understand and has displayed what it does want and understand. It is working exactly as it should.

If you don't like the way your Raymarine displays the data that you feed into it you need to ask Raymarine how to suppress either the import or display. Alternatively, don't give it data that you don't want it to display.

You have been shown using multiple tools that it is a trivial exercise to strip out information that your Raymarine understands and wants to show you but that you don't want to see.

The fact that we have been having this conversation over the course of several days would certainly seem to argue against your assertion that it seems "pointless to suggest something to the OCPN crowd". It seems to me that that crowd has been very helpful and has given you multiple solutions to your problem.

[wholybee]

Originally Posted by fgd3

I think you misunderstood "consumer". I believe "consumer" refers to the device which will be using the data, not the person who directs the data to the device. The point is that OpenCPN complies with the standard for creating GPX files. If other devices fail to conform to that standard why should it be incumbent on OpenCPN to diverge from the standard in order to compensate for the failure of other devices to properly process properly formatted GPX data?

No misunderstanding.

Conforming to the standard only means that it conforms to the standard. The GPX Schema describes the format of the gpx file. The type of data it can contain, and how to describe it. It does not specify that all that data *must* be present, or how an application would use or display that data.

For that, there is no standard. Every application handles the data in its own way. It is unfriendly to assume that just because you adhere to the "standard", your way is correct and it is up to the other application to make any needed changes for interoperability. The other application is adhering to the standard as well, and perhaps is just as arrogant in thinking that because they adhere to the standard they should change.

Consider the Predictwind gpx exports, which include lots of extraneous (but still conforming to the standard) data. By this reasoning, it is up to OpenCPN to display it correctly, and it does not.

I have had this issue as well, and use another program to convert the data. GPSBabel was written just for this, because nearly all programs adhere to the standard, and yet few work together.

I have to side with the OP here. A primary goal of any developer should be to make the program easier and work better. This would be a significant improvement, in line with an older thread asking how to improve interoperability. I might not make it a priority, but to just say "no" because the developers feel they comply with the standard and because there is a time consuming workaround involving 3rd party software is disappointing.

I feel like more and more the developers are ignoring user requests and taking the application in their own direction. It is their hard work and they have that right, but it is disappointing to see requests like this hit a hard no and the user basically be told they are wrong, time and time again.

At least put it in flyspray (Do you still use flyspray?) and give it a chance. Put it on the list for 6.0 or something. Or, maybe a programmer that isn't a core developer might like to work on it. I might like to look at or work on the feature. It would take a lot of effort to learn the structure of the code and how it works, but I might be able to do that.

To follow with Nohal's reasoning given for another recent "No" answer, maybe a Plugin can be developed for GPX export/import that is more robust and has these options? A few simple checkboxes on the dialog in the main application seems easier, but a plugin to do the export would work also.

[nohal]

Well... Predictwind includes a name with the route points. As we were told in this thread, that is absolutely useless and should not do it
And OpenCPN displays it by default exactly as the Axiom MFD does. So at least we are same broken, if we are broken at all.

I may just assume that "OpenCPN is not displaying GPX exported from Predictwind correctly" translates to "I think that what Predictwind exports is a valid GPX OpenCPN misreads and I do not like that OpenCPN displays point names if not instructed otherwise in the GPX".

So let's have a brief look at both parts of that, starting with the first
What Predictwind exports, at least what it exports for me, is certainly not even valid XML conforming to the GPX schema. Why? Because lat and lon are defined like this

...
<xsd:attribute name="lat" type="latitudeType" use="required">
<xsd:annotation>
<xsd:documentation> The latitude of the point. Decimal degrees, WGS84 datum. </xsd:documentation>
</xsd:annotation>
</xsd:attribute>
<xsd:attribute name="lon" type="longitudeType" use="required">
<xsd:annotation>
<xsd:documentation> The latitude of the point. Decimal degrees, WGS84 datum. </xsd:documentation>
</xsd:annotation>
</xsd:attribute>
...
<xsd:simpleType name="latitudeType">
<xsd:annotation>
<xsd:documentation> The latitude of the point. Decimal degrees, WGS84 datum. </xsd:documentation>
</xsd:annotation>
<xsd:restriction base="xsd:decimal">
<xsd:minInclusive value="-90.0"/>
<xsd:maxInclusive value="90.0"/>
</xsd:restriction>
</xsd:simpleType>
<xsd:simpleType name="longitudeType">
<xsd:annotation>
<xsd:documentation> The longitude of the point. Decimal degrees, WGS84 datum. </xsd:documentation>
</xsd:annotation>
<xsd:restriction base="xsd:decimal">
<xsd:minInclusive value="-180.0"/>
<xsd:maxExclusive value="180.0"/>
</xsd:restriction>
</xsd:simpleType>
...

and xsd:decimal clearly does not allow comma as a separator, as shown for example here: XML Schema 1.0 xsd:decimal - Complete documentation and samples

I would probably not call this a problem of OpenCPN, but a prime example of how the producer is breaking workflows by NOT adhering to the standard and exporting broken, non standard compliant data. But sure can't stop you from calling it otherwise.

Fortunately this is very simple to fix in your workflow, for example using

sed -i -e 's/l\(..\)="\(-*[0-9]*\),\(-*[0-9]*\)"/l\1="\2.\3"/g' <PREDICTWIND_ROUTE.GPX>

The other part is sure matter of personal preference, but even easier to tailor to yours, using for example

sed -i -e 's/<extensions>/<extensions>\n<opencpn:viz_name>0<\/opencpn:viz_name>/g' <PREDICTWIND_ROUTE.GPX>

So at the end we have a two line script and a happy you. And exactly the same applies to the OP's request.

[wholybee]

Originally Posted by nohal

Well... Predictwind includes a name with the route points. As we were told in this thread, that is absolutely useless and should not do it
And OpenCPN displays it by default exactly as the Axiom MFD does. So at least we are same broken, if we are broken at all.

<snip>

Note: Predictwind doesn't use a comma for me. I believe this is a known bug related to locality settings. I have heard of it before from European users.

But Wow. My point completely went by you. I don't actually think either OpenCPN or Predictwind are broken. I was merely applying your own reasoning, that if a gpx file includes data that isn't needed, it is up to the consumer to only use the data that is required. So Predictwind is "correct" by including as much information as it can, and it is OpenCPN's "problem" to display all of it instead of only what the user wants. That is the reasoning you applied to the Raymarine issue, No? You didn't want to remove "important" data from the GPX file export because Raymarine should be able to just not use it. It seems that the correct answer for you is that it is always up to the other developer to make a change, and never a change that should be made in OpenCPN. Or else, the user should just live with it and come up with their own workaround.

I am flabbergasted that you would suggest to casual Windows users that they should write a script using regex instead of making a change to the application. I did not expect that answer. Your doing that only proves my point. I even offered to give it a shot to help make the changes to OpenCPN myself. I am just dumbfounded at the arrogance of your suggestion, and the complete dismissal of someone willing to help you guys out. I was planning on spending my lunch break familiarizing myself with the code, but I guess that isn't welcome.

[nohal]

Sorry for missing your point, but I haven't really seen any. You are of course welcome to implement those "few checkboxes", nobody can prohibit you from that and if what you do makes sense, I have absolutely no doubt that your pull request will be accepted.

[derfy]

Tell you what, guys...

I'm going to come back with a Python program that converts all the GPX files in a directory to scrubbed GPX files that contain only route names and route point lat and lon.

I'll write the code myself and post it on this thread.

The name of the program will be RudeGPX or maybe GPXAnarchy

Wrote my first code in 1972, so I thnk I can figure it out.

Just like everything on a boat, if you want it fixed right, you gotta DIY. Gonna start on it right after I finish re-coring the wet balsa on my aft coach roof. Should I use epoxy or polyester?

[derfy]

I was able to generate this code in about 30 minutes this morning with ChatGPT. I will report back when it is debugged.

RudeGPX is under development. Cheers..

[wholybee]

Originally Posted by derfy

I was able to generate this code in about 30 minutes this morning with ChatGPT. I will report back when it is debugged.

RudeGPX is under development. Cheers..

ChatGPT is pretty incredible. I have had it write only a few very simple things, mostly to test it. But it is a great reference, or to have it review code I can't get working.

I have been using GPS Babel for the conversions.

@nohal

I want you to know that even though we spar here about OpenCPN once in a while, I do have a lot of respect for you, and I hope you don't take it personally.

I looked though the related code a bit yesterday, and I think I am going to tackle it. I at least see how it works now and the relevant part isn't too complicated. If not successful, at least it will be a learning experience.

[derfy]

Here is a working Python script... RudeGPX_v1. Finished it during lunch.

It's my contrib code to this thread and group. Hope you like it. Use it as your own risk. No warranties are offered or implied. No property rights are granted. The code is mine, but you can use it. Report bugs and features on this thread.

I had to attach the script as an MS Word doc since txt files are limited to 1000 bytes on this forum. Save it as a text file and give it the extension .py.
If you have trouble, I can post the .py file somewhere on the web.

This script scrubs GPX routes in .gpx files of all info elements except the Route Name and lat and lon of each route point. I leaves Waypoints and Tracks alone.

To use it you need Python and a library called gpxpy on your PC. These are free from Microsoft but have to be installed. It's a 20 minute deal and a reboot. If anyone cares, I will do a little "how to".

To scrub one or many files, just click on the attached .py file. It launches python and asks you to enter the path containing the input .gpx files (the ones you want to scrub), and the path where you want to put the output (scrubbed) files.

Then it runs.

I cleaned up a single file with 96 routes, thousands of route points. The original file from OpenCPN was 1.3 Mbytes, the output gpx is 127 kbytes (10x smaller).

Input routes looked like this (first few route points)....

<rte>
<name>ALLI2ALLIB</name>
<extensions>
<opencpn:guid>69deffff-1dc6-40d9-ac26-5d2071fcffff</opencpn:guid>
<opencpn:viz>1</opencpn:viz>
<opencpnlanned_speed>6.00</opencpnlanned_speed>
<opencpn:time_display>PC</opencpn:time_display>
<gpxx:RouteExtension>
<gpxx:IsAutoNamed>false</gpxx:IsAutoNamed>
</gpxx:RouteExtension>
</extensions>
<rtept lat="35.671527000" lon="-76.053428000">
<time>2023-03-26T17:30:32Z</time>
<name>001</name>
<sym>square</sym>
<type>WPT</type>
<extensions>
<opencpn:guid>3bcb0000-baa3-44b4-8576-8b6d7eeeffff</opencpn:guid>
<opencpn:viz_name>1</opencpn:viz_name>
<opencpn:arrival_radius>0.050</opencpn:arrival_radius>
<opencpn:waypoint_range_rings visible="false" number="0" step="-1" units="-1" colour="#FFFFFF" />
<opencpn:scale_min_max UseScale="false" ScaleMin="2147483646" ScaleMax="0" />
</extensions>
</rtept>
<rtept lat="35.669666000" lon="-76.042172000">
<time>2023-03-26T17:30:32Z</time>
<name>003</name>
<sym>square</sym>
<type>WPT</type>
<extensions>
<opencpn:guid>554affff-d23b-4439-9c0f-b5a90b140000</opencpn:guid>
<opencpn:viz_name>1</opencpn:viz_name>
<opencpn:arrival_radius>0.050</opencpn:arrival_radius>
<opencpn:waypoint_range_rings visible="false" number="0" step="-1" units="-1" colour="#FFFFFF" />
<opencpn:scale_min_max UseScale="false" ScaleMin="2147483646" ScaleMax="0" />
</extensions>
</rtept>
<rtept lat="35.670003000" lon="-76.038408000">
<time>2023-03-26T17:30:32Z</time>
<name>NM001</name>
<sym>square</sym>
<type>WPT</type>
<extensions>
<opencpn:guid>632dffff-541a-45b6-beee-7d8007f50000</opencpn:guid>
<opencpn:viz_name>1</opencpn:viz_name>
<opencpn:arrival_radius>0.050</opencpn:arrival_radius>
<opencpn:waypoint_range_rings visible="false" number="0" step="-1" units="-1" colour="#FFFFFF" />
<opencpn:scale_min_max UseScale="false" ScaleMin="2147483646" ScaleMax="0" />
</extensions>

The scrubbed Output routes look like this...

<rte>
<name>ALLI2ALLIB</name>
<rtept lat="35.671527" lon="-76.053428">
</rtept>
<rtept lat="35.669666" lon="-76.042172">
</rtept>
<rtept lat="35.670003" lon="-76.038408">
</rtept>

[derfy]

Originally Posted by wholybee

ChatGPT is pretty incredible. I have had it write only a few very simple things, mostly to test it. But it is a great reference, or to have it review code I can't get working.

I have been using GPS Babel for the conversions.

@nohal

I want you to know that even though we spar here about OpenCPN once in a while, I do have a lot of respect for you, and I hope you don't take it personally.

I looked though the related code a bit yesterday, and I think I am going to tackle it. I at least see how it works now and the relevant part isn't too complicated. If not successful, at least it will be a learning experience.

Thanks. I have huge respect for you and all the developers of OpenCPN. It is amazing.

Please see my previous posting for contrib code. ChatGPT blew me away. It wrote the code.

I just like to drive boats and use this stuff. I don't do tech anymore.