Why GUID ?

[fgd3]

In exported routes (.gpx) I noticed that every route and every waypoint within a route has its own Globally Unique Identifier (guid). What's the purpose of all these guids? One benefit I can see is that if you import a route that someone else gave you OpenCPN will recognize it is a different route even if it happens to have the same name as one of your existing routes. Is that it or is there more to it?

Fabbian

[RainDog]

GUIDs are normally used by programmers any time they want to avoid a name collision. They are used because they take all the work out of something that can be a lot of work to get correct. I would guess the developer used them to avoid the situation you describe.

[nohal]

Correct. GUID stays for globally unique identifier - that should say it all. While the need for unique identification of routes and waypoints could seem as an easy task, especially in single-user environment - it's not even there. If you get to services connecting people from around the globe, it starts to be a nightmare. With the guid for all the navobjects we are prepared for such a situation. That was actually the point (beside some improvements and simplification to the internal logic) I was following while writing the new GPX implementation for 2.3

Pavel

[PjotrC]

I do not know the exact policy for managing the domain of Nav Objects.

It is clear that all things having equal GUIDs should better be considered the same thing - this is the purpose of allocating unique identifiers.
Usually, the GUID is the _primary_ key, in OpenCPN it seems to be a _secondary_ key.
Decision whether to allocate for something a new GUID or to associate it with an existing one, based on equality of attributes, is a matter of policy.

Whatever the policy is, I am not sure that we can or should extend its reach outside of OpenCPN processing environment.

GUIDs identify concrete objects within specific namespace.
In another namespace, or under another policy, an identical object might be another thing, or an object with slightly (in the given metrics) differing attributes could be considered the same.

In other words, it sounds OK that GUIDs are used inside dynamic data structures, and that they are used in files and communication channels where the given GUID allocation policy holds,

I am not sure that - as a rule - GUIDs should be used everywhere.

Half of this is presently implemented, since there are two versions of importing everything - with GUIDs (treated as secondary) or without GUIDs.
However, all objects are exported with GUIDs.

I think there should be two versions of export as well: with GUID or without GUID, depending on whether we mean to interpret the exported data as concrete objects, or just patterns of attribute values.

A proposal:

A Save/Restore (with GUIDs) would operate within OpenCPN policy scope (this is definitely navobj.xml and these would be the backup files).

An Export/Import (without GUIDs) would be for environments, where the assumed object unification policy does not hold.

Of course, technically it can be done with simple text postprocessing, removing the GUID lines from GPX files, but it would be far better to implement the model explicitly ...

Example:

When building a route passing by a location of a Safe-Water Buoy Mark, the user is asked "Use the nearby waypoint?" The policy is to honor the answer, regardless of any attribute values the waypoints might have.

Without this user input it is impossible to decide, whether to share or clone the point.

A policy might say, that if the nearby waypoint has a name other than an automatic number, we share it, otherwise we clone it.
Depending on all this, zero or one new GUIDs are issued.
Imagine the user said YES, SHARE IT.
The route and the waypoint is saved with an identical waypoint GUID in them.

All is fine as long as we operate within this policy's enforcement capabilities.

The moment we send both routes into the wild, things break.

The route is sent to Ann, who edits the tricky point, moving it to North.
The route is also sent to Barbara, who moves the point to South.

After completing their respective circumnavigations, both routes return to us, and we import them.
Which of the points, if any, is good and which is bad?
In principle, the user should be asked again to choose... that should happen with Save/Restore.

In 2.4.x, we reasonably assume that the guy who was here first takes the floor, but even then we get two or perhaps three clearly different points with the same GUID. How to interpret this? Will they be merged again if all the attributes meet equal some time in the future? I guess not.

The point is, that once we know that we can no longer enforce the policy, we should not issue GUIDs nor respect them automatically, if present.
This should be called Export/Import.

Seems a safe step to take, regardless of Policy.

Another step would be to verify integrity of GPX files and remove GUIDs where results are doubtful.

Or am I getting it wrong?

[fgd3]

Originally Posted by PjotrC

I do not know the exact policy for managing the domain of Nav Objects...

You raised some excellent points. There is a belief that two objects with the same GUID are identical. That's certainly consistent with the concept of a GUID. There are two reasons why that belief is incorrect.

First, given the specifications for GUIDs, the best you can say is that two objects with the same GUID are very, very, very likely to be identical. The probability approaches certainty, but doesn't reach it.

Second, errors may cause different objects to be assigned the same GUID. There are programming errors that cause GUIDs to be calculated incorrectly, like the bug a couple of OpenCPN versions back that required us to manually remove the GUIDs from our .gpx files. There are conceptual errors where one object is replaced with a another but the existing GUID is retained.

In your example of the route sent to Ann (who edits it by moving a point north) and Barbara (who edits it by moving the same point south) the failure occurs when the program does not assign a new GUID each time the route is edited. After all, it can't be the same route if it goes to a different place.

You can simulate the situation you described by following these steps:

1. Drop a mark, then name it "Test Mark".
2. Create a route which incorporates "Test Mark".
3. Name the route "Test Route" and export it as "Test1.gpx".
4. Drag "Test Mark" to the north.
5. Export the route as "Test1N.gpx"
6. Use the Route Manager to delete all routes and waypoints.
7. Import "Test1.gpx"
8. Drag "Test Mark" to the south.
9. Export the route as "Test1S.gpx"
10. Import Test1.gpx.
11. Import Test1N.gpx.

You should have three routes on your display, each with a common start and end point and each passing through a different "Test Mark". If you delete all the routes you will be left with three marks having the same name. If you look in the .gpx files you'll see that all three routes and all their route points are the same from file to file. Does anyone think that is the way it should work?

What exactly is the problem here? A user may, if he wishes, create any number of waypoints with the same name. It's not a good idea, but the program allows it. He may also create any number of routes with the same name. Also not a good idea, but it's the default condition (the name being "Unnamed route").

The problem isn't that there are three different waypoints or three different routes with the same name. The problem is that we have different objects with the same GUID, breaking the promise that objects with the same GUID are the same object.

OpenCPN retains routes and points until they are explicitly deleted. I assume their GUIDs are saved with them. Is that good enough for your proposal? If so, the only other step required would be to remove GUIDs from the .gpx files.

Fabbian

[PjotrC]

Originally Posted by fgd3

OpenCPN retains routes and points until they are explicitly deleted. I assume their GUIDs are saved with them. Is that good enough for your proposal? If so, the only other step required would be to remove GUIDs from the .gpx files.

Fabbian,

Yes, the short version of the message is: "Do not include GUIDs in exported GPX".

But Pavel is right, that in an interconnected world things will get more complex. It may be only a matter of technical detail, if Ann and Barbara are using their local copies of The Route, or they are remotely accessing The Route held e.g. in Zoe's Repository. They might not even be aware of where it is.

With this perspective in sight, just stripping GUID from GPX will not be enough.

A systematic solution will be needed then.

Cheers,

Piotr

[fgd3]

Originally Posted by PjotrC

Fabbian,


But Pavel is right, that in an interconnected world things will get more complex. It may be only a matter of technical detail, if Ann and Barbara are using their local copies of The Route, or they are remotely accessing The Route held e.g. in Zoe's Repository. They might not even be aware of where it is.


Piotr

Until a more systematic approach is developed, it seems clear the GUID should be removed from .gpx files.

The .gpx format is intended as a common format that can be used by many programs. The GUID extension is specific to OpenCPN. So Ann sends her copy of the route to Carla, who displays and edits it with a different program. That program is exceptionally well-behaved, so it preserves the OpenCPN extensions even as it adds its own extension with a GUID it has assigned. Carla uploads the route to Zoe's repository which uses yet another program. Zoe's program ignores extensions it doesn't recognize, so both the prior GUIDs are discarded. The program creates a new GUID in a new extension. Barbara downloads the route from Zoe's repository. OpenCPN doesn't recognize the GUID created by Zoe's program, so it discards it and creates a new one with an OpenCPN extension. What difference does it make? Barbara is going to look at the route and say "This isn't right, the important point is a mile north of where it should be!" She drags the point to where she wants it. What did the various GUIDs do for anyone during all this?

I know Pavel is trying to look ahead but I think he has taken a wrong turn and needs to give it more thought.

When the global communications infrastructure is good enough that ocean navigators can rely on data from a repository whose location they don't know (instead of their own local copies) I don't think we're going to be using GUIDs issued in a program-specific extension to keep track of resources.

Fabbian

[nohal]

Fabbian...
Enough. The guid is in the extensions section which is there exactly for this reason, it has absolutely no influence on other programs using our GPX and is 100% according to the standard.

Howgh

Pavel

[PjotrC]

Originally Posted by fgd3

Until a more systematic approach is developed, it seems clear the GUID should be removed from .gpx files.

Yes, similar scenarios can be easily found using OpenCPN only...

The extended version of the message is:

1. GUID is not useful in gpx import/export, especially if conflicts are not detected and handled on import.
2. They might be useful in navobj save/restore, once there is a policy for handling them.
3. Once GUIDs are issued, they should be the primary identification.

In the code I write now, I tend to use GUID equality rather than pointer equality as the test for identical objects.

Piotr

[nohal]

Originally Posted by PjotrC

In the code I write now, I tend to use GUID equality rather than pointer equality as the test for identical objects.

Piotr,
you do very well. What has to be kept on mind is that the scope of our current implementation is identifying navobject in a single OpenCPN instance. GPX-wise, outside of this scope, there is (and will not be until we introduce such a mechanism ourselves) no way to assure the integrity of the information (although it does work in many cases, it of course does not in many others, but both are nothing more than a side effect currently). If we want to extend this scope, we have to rethink and change the way we handle stuff, starting with treating Save/Load and Import/Export as different scenarios, which they are. Until we decide to do that, there is no way to satisfy every possible use case...
One of the first questions that will arise: if I have more than one computer, am I just Ann or am I Ann and Beti? Depends on the POV, I would say

Pavel

[nohal]

Originally Posted by PjotrC

1. GUID is not useful in gpx import/export, especially if conflicts are not detected and handled on import.

They are detected and handled - the one coming later is changed. Product of mixing import with load. If someone has a better idea, it's welcome...

[PjotrC]

Originally Posted by nohal

the one coming later is changed.

But it is the GUID that is changed... isn't it? Which means GUID is not an identifier outside of the single run-time instance of OpenCPN domain (well: assuming nobody is transferring navobj.xml between instances - a single installation).

The better idea is just not to use GUIDs if they are not primary identifiers. Skip them from gpx. Save megabytes.

[nohal]

Scenario:
I create routes that share some waypoints and do not share others (the only difference is that I said "use the nearby waypoint" or not, otherwise, they have the same parameters) - dont ask why, it's the same artificial thing as all the other examples in this thread. Export them without guids. (Delete them in routemenager) Import them. The sharing info is lost, no way to recover it. For me bug. That's why the guid is always there.

Until we have separate save/load and import/export logic, then I will be the first one not exporting the guid or other product-specific extensions.

What about the other opencpn extensions? Should we also save those megabytes they take? We can dump them, the gpx will still be valid. More of our information will be lost. Good idea? I simply think it isn't. You may think differently.

[PjotrC]

Originally Posted by nohal

Scenario:
The sharing info is lost, no way to recover it. For me bug. That's why the guid is always there.

Until we have separate save/load and import/export logic, then I will be the first one not exporting the guid or other product-specific extensions.

What about the other opencpn extensions? Should we also save those megabytes they take? We can dump them, the gpx will still be valid. More of our information will be lost. Good idea? I simply think it isn't. You may think differently.

Pavel,

The sharing is lost Forums - Post 644530">GUID-IRJDSUNE9932123321222xxeww-or-IRJDSUNE9932123321222xxeww-no-IRJDSUNE9932123321222xxeww-GUID. The point is that the presence of GUIDs without the policy and means to enforce it is harmful. All three factors must be present together, or none.

The effect of other OpenCPN-specific attributes is very local, so less fallout there. I would probably prefer the export version aimed at maximum portability, so all reasonable defaults only, and the save-restore aimed at internal OCPN usage - as detailed as it needs be. However, as a commited GPX-by-hand editor, my life was much easier with the single-line <prop:A,0,0,1,1>

No need to rush... for layers import, I just skip GUIDS, but I make this clear in the policy .

Cheers,

Piotr

[nohal]

Yes, I'm aware of the routepoint sharing bug and will fix it as soon as I have time. By the way, don't forget to file a feature request for removing the layer navobjects from the .changes to make me not forget
I agree 100% with the rest - we should separate saving/loading and import/export, until then our policy simply can't be clear enough, or we have to loose part of the functionality.
<prop:....> was nice. And a total headache to handle and extend across versions But whatever we have, sed does the trick whatever you throw it to crunch

Pavel