Datastream Priorities

[bdbcat]

Hello testers...

The current implementation of Datastream priorities in 3.1.1105 is not really working as one might expect. It needs re-thinking. See, for example,
OpenCPN::Tracker All Projects: Tasklist

Currently, the priority applies to a specific stream, or source.

So, here's a potential problem. Say you have two streams, both with RMC. You want to use stream A for navigation, since it is attached to a "better" GPS receiver. But, you want to revert to Stream B if stream A dies. OK, fine. Set Stream A priority higher than Stream B.

However, for example, your only source of HDG is coming on stream B. Stream B is blocked, as being lower priority than A. You will never get the HDG message until Stream A dies. Not the desired result.

So, the question becomes:
"Do priorities apply to streams, or to messages, or to specific messages on specific streams?"

The third option will of course lead to a very complex 2-dimensional UI...
We will need to establish and configure the following:
For all possible message types, which stream should take priority in the case of duplicates messages within some smallish timeframe?

As a general query, then:
How are users desiring to use priorities among multiple NMEA streams?
What problems are we trying to solve here?

Head spinning....
Dave

[ChuckSK]

Dave

A fairly simple solution would be to have all the streams sent to the core navigation code. The nmea messages with the highest stream priority would be selected for use by the core. A nmea message that was not desired to be used but came from a high priority stream would have to be blocked from the stream.


Chuck

[Paul Elliott]

I'm not promoting NavMonPc (which I wrote), but as an example it does prioritization on individual message types / input ports. I do have wildcard routing defaults (priority is one aspect of routing) which are port-based, but individual messages from specific ports can be given explicit routing parameters.

I suspect most people just use the wildcard routing, but on my boat I do use some explicit routes when I have multiple sources for the same message-type. For example, my chartplotter sends me GPS data that is better than my AIS transponder's GPS, so I use the chartplotter's data when it's available. I often turn off the chartplotter and then I want to use the AIS transponder's data. Both of these sources are run through a NMEA mux, and then to my computer, so I can't do this with just port-based routing. Fortunately, the chartplotter and transponder send slightly different prefixes on their NMEA data so my router can tell them apart. (Or do I have the mux modifying the prefixes? I don't recall.)

[hoolie]

I think the output stream will also need to be thought through again.

On this beta 3.1.1105 I can't send route data to my Garmin GPS because, presumably, it's not recognised as a GPS. It's a Garmin protocol serial connection that OpenCPN can't handle, so it is basically ignored by the connections module.

I can cope with the input being ignored - it's always been like that (though it would be nice if it wasn't) - but I DO need to define a data path between the route manager and my serially connected GPS. I can also envisage system configurations where you might want to send this data to more than one external device, perhaps with different protocols.

[bdbcat]

Chuck...

I don't immediately see how to resolve the time ambiguity, as NMEA messages arrive asynchronously.

For example, if we get RMC from Stream A at priority 1, we have to use it now. We can't know that we might get an RMC from Stream B at higher priority 2 in the very near future, implying that we should not have used RMC from Stream A at all.

So then maybe we need some one-shot logic and a history buffer. This would need to be on a per message-type basis. That is, if I am receiving Priority 2 RMC messages in a timely manner, then ignore RMC from priority 1 and 0 Streams. But continue to process other messages from Priority 1 Streams. If the Priority 2 RMC messages fail after some (variable) time period, then fall back to Priority 1 RMC, if available.

And so on for other interesting (all?) message types...
I see some logic to this, and I will try to implement it...

I agree that it is sensible to block unwanted messages from high priority streams, so that lower priority stream messages will get through, assuming the above logic can be implemented.

Also, Paul, thanks for the input. My instinct is to shy away from an overly complicated UI for setting per-message priorities, but to try to make it "do the right thing" out of the box.

Am I making this too complicated?

Dave

[bdbcat]

hoolie....

Garmin protocol is on the todo list. The output of routes, etc. is easily lifted from pre-3.1.1105 code base. Just got left off of 1105 in the interests of getting the Beta out there to find the first-order bugs.

Garmin protocol for input is also on the list. I am waiting to receive in the eBay mail a Garmin eTrex (serial) model that allegedly talks pvt format, after discovering that my ancient Garmin GPS38 does not....Maybe ready for the next Beta. I agree that having to switch your GPS from NMEA mode to GRMN mode for route upload could become an aggravation over time.

Just to confirm, is your Garmin GPS a serial port, or USB model?

Dave

[Paul Elliott]

Originally Posted by bdbcat

Also, Paul, thanks for the input. My instinct is to shy away from an overly complicated UI for setting per-message priorities, but to try to make it "do the right thing" out of the box.

You're welcome, and thanks for working on a very valuable program. I hear lots of good things about it.

I agree with keeping it simple. I try to keep the special cases to a minimum, which means that everything is run through a pretty complicated router. In the vast majority of cases the fancy stuff isn't required, but I wanted the flexibility.

As for NMEA message arrival time race conditions, I have "message active" timers for my decoders, with individual timers for each of three priority levels. This lets me take data from an active high-priority source, and ignore lower-priority data until the high-priority source goes quiet (five second time-out if I recall).

Once I have the mechanism in place for all this, it's trivially easy to design for three (or more) priority levels. Multiple decoders do consume memory though, but not much more data is actually being stored, and most of the decoding logic is shared.

I have no idea if this type of complication would be justified in OpenCPN. I personally have only one or two sources of duplicate NMEA message-types, so there's probably an easier way to deal with this. It would be great to get feedback from all the OpenCPN (and NavMonPc) users, rather than just from the involved users. I assume that the quiet ones are happy enough with the status quo, or have given up in disgust.

[hoolie]

Originally Posted by bdbcat

hoolie....

Garmin protocol is on the todo list. The output of routes, etc. is easily lifted from pre-3.1.1105 code base. Just got left off of 1105 in the interests of getting the Beta out there to find the first-order bugs.

Garmin protocol for input is also on the list. I am waiting to receive in the eBay mail a Garmin eTrex (serial) model that allegedly talks pvt format, after discovering that my ancient Garmin GPS38 does not....Maybe ready for the next Beta. I agree that having to switch your GPS from NMEA mode to GRMN mode for route upload could become an aggravation over time.

Just to confirm, is your Garmin GPS a serial port, or USB model?

Dave

OK - thanks, it's work in progress!

My GPS152 uses a serial port and has no USB capability. I use it as a cockpit instrument and have a BU373 puck for the PC, so I'm more concerned about data transfer to the GPS than from it.

Incidentally, I have never had any problems with the BU373 on WinXP/sp3 though I haven't yet been able to fully test 3.1.1105.

[ChuckSK]

Dave

Would a history be necessary? If each decoder checked for stream priority and passed only the sentence from the highest priority stream but reset the passing priority to zero on a watchdog timeout, a timed-out high priority sentence would simply be overwritten by a new lower priority sentence and a new passing priority would be established.

This would require saving a stream priority for each sentence type. And maybe modifying the watchdog timer for each sentence type.

Chuck

[bdbcat]

Chuck...

Bingo. I came to the same logic as I wrote the code. You will see it soon.
In my implementation, we do not need to have a watchdog to set priority to zero on timeout. Its enough to say that when a lower priority message arrives, and the timeout period has expired, then use the lower priority stream for this message, and set the current priority thus. If the lost stream later reappears at higher priority, it takes over again. All this logic occurs at the sentence type level, not just the stream level.

So, the priority scheme does two things:

1. Resolves duplicate message types (e.g.GPRMC) for multiple streams...
and
2. Imposes some logic on substitution/replacement policy for lost streams.

Dave

[bdbcat]

Folks...

Here is a corner case that my current logic will not handle.

GPRMC from a priority 1 channel.
AIVDO messages from an AIS transponder connected to a Priority 0 channel.

What you might want is for the GPRMC to be the primary positioning input, and for the transponder to take over if the primary fails.

Won't work that way in the current thinking, since the message types differ.

Same would be the case if primary was GPGLL, and secondary was GPRMC.

Could be fixed, of course, at the cost of a lot of added complexity in the code. We would need to shift our thinking from specific NMEA message types to a focus on actual message content, understanding that NMEA messages often omit some content if the talker does not have that info.

Is this scenario very important to anyone?

Dave

[SethDart]

Hi,

That's what is done in the dashboard plugin. I have internal sentences for SOG COG that don't have a unique NMEA source. The priority thing (with no exipration yet) inject the value from the "best" nmea sentence received. This had to be done for not-nmea source (gpsd position) as well.

IMHO, the filtering logic applies to NMEA, not converted sentence or multiple input sentences. The current behaviour is good to me. Even with corner cases like you described. You'd also have corner cases with plugins injecting NMEA (so without priority set by user. Not tested but you'll probably end up with a bouncing boat if you have NMEA input and the VDR plugin playing some nmea data)

BTW, can plugins list datastream inputs and push to a specific one? I'd like to create a plugin to manage SRT AIS boards, it may require ability for the user to select the corresponding port and for the plugin to push data to that port.

Thanks,
Jean-Eudes