Zeus3 NEMA fastpacket errors

[Paul L]

Originally Posted by tanglewood

....... Or maybe a receiver expects in-order delivery and doesn't get it? I don't know if reordering is a characteristic of canbus.

.....

I don't think the bus can get out of order packets but some Linux canbus stacks can deliver packets out of order. If software is building up a fastpacket with a timeout and it sees an out of order packet, it will probably assume a lost packet and fail the fastpacket. If I remember correctly the out of order case is very much a corner case under a busy network stack.

[Unity]

Tanglewood - thanks for a great description! I think I'm starting to see your point here … simply that why would you have FP errors without seeing RX/TX errors? How culd you have all those "missing packets" without having the majority also show up as RX/TX failures?

Now its possible that Navionics doesn't track RX/TX errors on FP as it's not required … but if you have network errors, then you would certainly expect to see those errors on other packets!

I'm on board with that … but I wonder then if perhaps there is another cause. By design rather than bugs or unintentionally side effects but inside some "prioritization" system that they have implemented. Since a FP consumes more wire time, if a "higher priority" message wants to be delivered from a transmitter who is currently sending FP, it would appear to be within spec to stop sending the rest of the FP and instead move onto sending the higher priority message (dropping the FP knowing that it isn't as important and a future transmission will accomplish it). One could argue in that case it is a feature (and maybe a good one) and by design and again not indicative of failure? Of course I'm just doing wild conjecture with no knowledge or citations!

Going back to the OP, would you agree the FP errors without RX/TX errors do not indicate a problem with the network but rather something the devices are doing themselves (for good or sloppy reasons)? If yes, then I think we are now contemplating if that is by design or by bug!

Maybe if other devices reported FP errors we could get insight into that.

Thanks for taking the time to share your experience / thoughts on this.

[Unity]

Originally Posted by Paul L

I don't think the bus can get out of order packets but some Linux canbus stacks can deliver packets out of order. If software is building up a fastpacket with a timeout and it sees an out of order packet, it will probably assume a lost packet and fail the fastpacket. If I remember correctly the out of order case is very much a corner case under a busy network stack.

That is another very interesting perspective on this, and also fits the evidence. That would fall closer to "sloppy programming" as the software should be designed to understand out-of-order can happen and queue up those messages until the delay expires or the full set is received... harder to do for sure, but would be the "right thing".

[Paul L]

Originally Posted by Unity

That is another very interesting perspective on this, and also fits the evidence. That would fall closer to "sloppy programming" as the software should be designed to understand out-of-order can happen and queue up those messages until the delay expires or the full set is received... harder to do for sure, but would be the "right thing".

Personally I think the stack should deliver packets in the order received off the wire, which it does in most all cases. No point in waiting for packet #2 If you receive #1 and then #3.

[Unity]

Agreed - either fix the stack or manage out-of-order at the app layer. Either way that explanation goes down to sloppy programming. Thx!

[tanglewood]

Originally Posted by Unity

Tanglewood - thanks for a great description! I think I'm starting to see your point here … simply that why would you have FP errors without seeing RX/TX errors? How culd you have all those "missing packets" without having the majority also show up as RX/TX failures?

Exactly, and more concisely stated than my ramblings.

Originally Posted by Unity

Now its possible that Navionics doesn't track RX/TX errors on FP as it's not required … but if you have network errors, then you would certainly expect to see those errors on other packets!

It is interesting that nobody else seems to accumulate and/or report FP errors, which further suggests to me that it's an event unique to the Navico implementation. Paul L provided an excellent example of how such a thing could be an error based on an implementation.

Originally Posted by Unity

I'm on board with that … but I wonder then if perhaps there is another cause. By design rather than bugs or unintentionally side effects but inside some "prioritization" system that they have implemented. Since a FP consumes more wire time, if a "higher priority" message wants to be delivered from a transmitter who is currently sending FP, it would appear to be within spec to stop sending the rest of the FP and instead move onto sending the higher priority message (dropping the FP knowing that it isn't as important and a future transmission will accomplish it). One could argue in that case it is a feature (and maybe a good one) and by design and again not indicative of failure? Of course I'm just doing wild conjecture with no knowledge or citations!

I think this will absolutely happen as part of normal operation. The NEMA paper pretty clearly states that other PGNs can and will be inter weaved with the FP component PGNs. The interwoven PGNs could be from other sources, or the same source as the FP. So reassembly, though still trivial, is more than simply taking the next N PGNs and assuming they are the rest of the FP. But a retarded implementation like that would certainly generate FP errors, wouldn't it?

Originally Posted by Unity

Going back to the OP, would you agree the FP errors without RX/TX errors do not indicate a problem with the network but rather something the devices are doing themselves (for good or sloppy reasons)? If yes, then I think we are now contemplating if that is by design or by bug!

Maybe if other devices reported FP errors we could get insight into that.

Thanks for taking the time to share your experience / thoughts on this.

Back when I had a Simrad based system, I was racking up lots of FP errors, and I did do some experimentation. Turning on and off the Simrad AIS device (NAIS400 as I recall) had a marked impact on the rate of accumulation of errors. That's no surprise since AIS PGNs are one of the more prolific FPs. I think the GNSS Sats in View is another FP message. So they are an ongoing part of daily life on a boat.


My money is on a brain-dead Navico implementation that drops FPs if they don't meet it's criteria, probably depending on some combination of the component parts immediately following the first packet, in order, and without intervening PGNs. And the reported error just gives a record of how many FPs are being dropped. Dropped FPs, like most other dropped PGNs, are not fatal since a new one is sure to come along soon after. But it's indicative of a sloppy implementation. I want my periodic instrumentation data on time, every time. Not just some of the time. As I have said in my blog, these are not web apps or phone apps where only your BFF will be annoyed if it fails. Our lives depend on our navigation equipment, and I hold it to a similar standard as medical equipment and airplanes.

[RobertJ]

Very helpful discussion, but I remain puzzled as to what conditions result in the network declaring a fast packet error. We have cases where hundreds of fast packet errors/day are logged with no apparent problems and no TX or RX errors. What am I missing? Thoughts?

[smac999]

Originally Posted by RobertJ

Basic symptom is one (of two) Zeus 3’s loosing Navionics chart and freezing up, sometimes an automatic restart occurs. Sometimes also shows radar scanner is lost.

charts and radar are shared via eithernet, not simnet. I think you are barking up the wrong tree.

[Unity]

Originally Posted by RobertJ

Very helpful discussion, but I remain puzzled as to what conditions result in the network declaring a fast packet error. We have cases where hundreds of fast packet errors/day are logged with no apparent problems and no TX or RX errors. What am I missing? Thoughts?

The conclusion is ignore fast packet errors. Only be concerned about a volume of the rx/tx errors. Hundreds of fast packet a day is consistent with others. If debugging an actual problem, be sure to look at all devices for errors. If one has very different numbers try removing it first. Otherwise it’s a binary-search game (remove half, if no errors try the other half, if errors remove another next half (now a quarter) and keep going until bad device / network point found). Of course do that only after checking all connections, terminators, and maybe even replacing the tee connectors since they are cheap.

[Unity]

Originally Posted by smac999

charts and radar are shared via eithernet, not simnet. I think you are barking up the wrong tree.

For sure the fast packets are not the OP issue. We fell down a (very interesting to me at least) rat hole trying to understand why we all get those FP errors anyway! Thx!

[RobertJ]

Yes we got sidetracked a bit, but a very interesting and useful discussion on those FP errors.
Coming back to the original problem, it is interesting that while the Zeus was freezing up, map transfer was corrupted, radar scanner not recognized, ...,the only errors logged were FP errors. The crash log contained numerous files but these cannot be read without a special utility that I don’t have (according to Navionics).
As to smac999’s comment—correct but the Ethernet and the simnet “mix” at the radar interface box, this facilitates MARPA and radar overlay on charts.
I will change out the problem Zeus over the weekend and we’ll see if the problem resolves.