Zeus3 NEMA fastpacket errors

[RobertJ]

For the last few months I have been chasing gremlins on my simnet network. Basic symptom is one (of two) Zeus 3’s loosing Navionics chart and freezing up, sometimes an automatic restart occurs. Sometimes also shows radar scanner is lost.
I have soft reset the plotter and simnet network and recently turned off the AIS on my B&G V50 as that has been reported to be a problem. Lastly I did a reset on the V 50.
Overall the problem has lessened significantly but not fully resolved.
One recurring sign is that the network accumulates fast packet errors repeatedly,
typically a few hundred over a day. No TX or RX errors, just fastpacket.
Are fastpacket errors common or is this an indicator that something is still very wrong?

[Paul L]

Sounds like you have a backbone issue. Check that both ends are properly terminated and no other drop is terminated. Look for connector issues..

[Suijin]

I second Paul’s advice as the first place to start. And make sure the connectors are TIGHT, turning both coupling rings. They can feel tight even when they are not fully seated.

I’m no expert but when I have had packet issues it’s always been a connector issue.

[Unity]

Agree - sounds like your NMEA bus is having problems. It’s not uncommon for those B&G “T” taps to be bad. I’d recommend getting new ones and swapping the old ones out (quickest and not too expensive) but you could always swap out just one or two and see if those fast packet errors go away.

If I recall correctly, the map data actually travels over the Ethernet connection. What all do you have I that? Are you using any third party devices like routers or other equipment that might cause IP address / routing issues?

[Unity]

One more thought - on a long bus the voltage drop can add up. If your failures occur more when the battery is lower / not charging then that could be an issue. On the Zeus3 you can edit the data panel to include voltage on the display and monitor it on both devices to see if it drops low (such as 11v or lower). Fixing is easy - add another power tap with fresh power (run the math on wire gauge/distance) to a point in the bus far away from the primary tap and nearer the furthest Zeus.

[RobertJ]

Thanks to all for your replies. I have checked the terminations and all seems correct. Yes the map data moves on the Ethernet as was stated. All equipment is B&G, new boat build completed late fall last year. There is a CZ zone device on the simnet that allows greater net flexibility—installed by the builder.
In the interim I talked with B&G tech support—their initial thoughts are that the plotter itself may be the problem—for two reasons: it’s the one plotter that shows the problems and the map data on the Ethernet vs. the simnet. They are sending me a replacement unit so we’ll see.
More to come as I continue to chase this ghost.

[Unity]

I got on board yesterday afternoon and turned on my electronics. This morning I checked and I have 297 fast packet errors but nothing else wrong. And my setup works without fail.



In many protocols with multiple device senders, it common to have multiple senders attempt to broadcast at the same time (collisions) which the protocol will detect and recover from. I’m guessing that that is what these errors on, and considering that it’s about one error every 2-3 minutes on a network moving a ton of traffic, that’s not at all bad by traditional networking standards. But maybe someone who understands the NMEA 2000 protocol can shed some light?

[RobertJ]

Unity thank you for doing that check, as you say these fastpacket errors seem to be normal on a multi-element network. My network load is typically about 25%, and over 2 days I had about 600 fastpacket errors but with no problems.
The two factors that seem to have made a difference are the soft reset on the Zeus 3 and turning off the AIS on the V50 radio. Jury is still out on whether the problems are resolved.

[Paul L]

Fastpackets are packets put together with multiple small data packets. Some low level of bus errors is acceptable. When I had excessive fastpacket errors it turned out to be a connector issue (of my own making).
It is pretty easy to determine which device(s) is involved by simply removing a device one at a time and running for awhile.

[tanglewood]

I had fast packer errors back when I had a Simrad EVO 2. I stripped it down to a minimal network, and both EVOs still generated errors. My conclusion is that it was one software bug, but I junked the whole package before tracking that particular problem down. I even looked at a display Simrad at a west marine and guess what? It had fast packet errors. So I suspect your network is fine and it's just a bug in the Zeus.

[Unity]

Paul L’s answer regarding FastPacket got me to dig in a bit deeper. Turns out these errors are normal and expected on any NMEA 2000 system and at low volume do not indicate any failure and are by design (not a bug).

Basically a N2K packet is small. Generally it carries only 8 bytes of payload. So when needing to send larger messages, one common method is “Fast Packet” which lets a transmitter rapidly send multiple 8-byte packets in a sequence. However, the protocol does not offer a collision/error recovery technique (such as one packet in a sequence being lost due to a layer 2 collision) instead assuming that such failures are non-critical and typically get sent again soon on the next update message.

There is another protocol called “multi-packet” that does offer error recovery but it comes at a cost - it is slower and consumes more wire time. So most devices that are broadcasting larger packets of information that is continuously being updated use Fast Packet, knowing that if it fails the next transmission will likely succeed.

Wikipedia has a great article on this: https://en.m.wikipedia.org/wiki/NMEA_2000. From the “Message Format...” section of that article, this is what they say about Fast Packet protocol delays compared to Multi-packet:

“Takes less time to send up to 223 bytes; no transfer protocol delays; no guarantee it is received by all nodes”

In summary - ignore fast packet errors. If you have a bus problem you will likely see it on the other protocol parameters such as under the various receive/transmit/protocol errors & overruns.

Chris

[tanglewood]

I actually think there is more to it than that, and that fast packet errors do indeed represent bugs, or at least sloppy programming.


Each normal size pack that makes up a fast packet is sent using all the normal arbitration mechanisms in CanBus, and as such those packets will not be corrupted, or will be detected as corrupted. As the article states, component fast packet messages are interspersed with normal messages, and it's up to the receiver to reassemble the fast packet elements into the larger message. If the individual messages are not getting through, or are being corrupted, there is a problem with the network and you would expect to see rx or tx errors irrespective of normal or fast packets.


I can only guess what a Navico Fast Packet Error actually signifies, but I suspect it means that the receiver was unable to fully reassemble a fast packet within some timeout period. Beyond that it's into the implementation details that none of us know.


BTW, I find it curious that nobody other than Navico reports fast packet errors. Has anyone seen any other products that do? I know I haven't, so it makes me wonder if it isn't something specific to Navico's N2K implementation.

[Unity]

I am no expert in N2K or CAN (though am quite steeped in other protocols), so you may well be right. Thanks to this discussion, I have done a bunch more reading and have learned a lot about N2K that will be helpful in the future to debug problems!

But what I am taking from my reading is that this is that this is a "feature" of the protocol, and would be experienced on all devices from any manufacturer that correctly adopts the protocol. While it's interesting that Navico decided to show it on the display and apparently other vendors have not, but that itself isn't an indication of anything about known failures vs. detailed diagnostics.

What I read (and is summarized nicely on that Wikipedia page, and collaborated by other articles including from nema.org) is that when using Fast Packets, the automatic CRC checking / handshaking is disabled. This means that bad packets would not automatically retransmit and instead the failure would be detected later as part of a missing packet in a fast packet sequence.

See the table in this section of the page, specifically the line:

CAN layer assures all (connected) nodes received the message and validated its CRC

where the Fast Packet column says

No handshaking

Reference: https://en.wikipedia.org/wiki/NMEA_2..._numbers_(PGNs).

An additional citation can be found here, from the official nema.org N2K document http://www.nmea.org/Assets/20090423%...mea%202000.pdf.

I take this to mean that packet errors would not be detected and retransmitted, and therefore packets within a sequence could be detected as lost. And supporting this is the entry in the same table where it says for "No guarantee it is received by all nodes".

So my conclusion from the above references is that Fast Packet errors are by design and expected, and should alone not be taken as a problem.

If anyone disagrees with this conclusion, perhaps you could provide some citations on this? I would love to read up on it to learn why my understanding is wrong!

Chris

[tanglewood]

As I'm sure you know, N2K is not only proprietary, but confidential as well. Anyone can take out a license and be let in on the secret, but in doing so, you agree to keep the secret. Because of this, I have intentionally NOT taken out a license, so I don't know definitively. But I have spoke with a number of people from a number of companies and been able to piece together quite a bit. Not perfect, but not bad either.


Canbus and J1939 on which N2K is build are both, in contrast, non-confidential. The docs are copyright and you have to buy them, but that's typical of industry standards . The key is that people in the know can talk openly about how things work. Not so for N2K.


Anyway, I diverge....


As I understand N2K, it is a datagram service only. There are a few exceptions, but they are just that - exceptions. And this makes sense when you consider that 90% of the N2K traffic is broadcast. So PGNs get sent and are protected by CRC. But there is no ack, no confirmation of delivery, and no retry.


However, because each PGN is protected by CRC, the recipient knows if it gets bad data. It doesn't know if a PGN gets totally lost in space, but if it comes through scrambled in any way, it can tell and that's what the rx and tx errors are reporting. It's a pretty safe bet that total packet loss will be preceded by packet corruption, so zero rx errors (the only thing you should ever see in a working network) pretty much assures there is also no PGN loss.


The fast packet protocol is just a simple way to link together a bunch of individual PGNs. But it is NOT a transport protocol with acks, retries, etc. It's just a series of PGNs that are sufficiently self-describing to be reassembled on the receiving side, assuming all make it through unscathed. The first PGN includes a count of PGNs in the fast packet, and each subsequent PGN includes it's place number in the fast packet. With that, the receiver knows which incoming PGNs are fast packet components, and can reassemble them before acting on them further.


The problem arises when not all component PGNs arrive within some time frame. The receiver doesn't know when the sender has finished sending, so it can only timeout if the fast packet never completely reassembles. This might be because of lost or corrupted PGNs, but I would expect those to be reported as rx errors in addition to FP errors. The only way I can see FP errors happening in the absence of corrupted PGNs is if 1) transmitters fail to send all component PGNs, the receiver doesn't have enough buffer space to reassemble all FP PGNs, or the receiver timeout is too short. Or maybe a receiver expects in-order delivery and doesn't get it? I don't know if reordering is a characteristic of canbus.


So at the PCN level, I don't believe there is any distinction between stand-alone PGNs and FP component PGNs. All arbitrate for bus access the same way, and all have the same protections and transmission characteristics.


And one thing to keep in mind is that canbus access arbitration can't scramble a packet. Arbitration all happens and is resolved, then the PGN gets sent while everyone waits. Part of why there are physical length limits are to ensure that the timing windows for all these steps are sufficient no mater where you are on the bus. It's much less of a free-for-all than the original CSMA-CD ethernet.


Oh and BTW, if you look at the NMEA paper, they show "no handshaking" under both single PGNs, and FP PGNs. So I think the wiki article is a tad sloppy with terminology and consistency, but otherwise a typically good article.

[tanglewood]

Originally Posted by Unity

I take this to mean that packet errors would not be detected and retransmitted, and therefore packets within a sequence could be detected as lost. And supporting this is the entry in the same table where it says for "No guarantee it is received by all nodes".

Agreed, but I am quite sure the same is true for all PGNs, not just FP PGNs. There is no loss detection, other than possibly timing out when you think you should be receiving something, like when reassembling a FP.

Originally Posted by Unity

So my conclusion from the above references is that Fast Packet errors are by design and expected, and should alone not be taken as a problem.

I would disagree with this. I blabbed on about it in my last post, but there should be zero CRC errors, and zero packet loss in a correctly working network. And I'm pretty sure it would be impossible for a packet to be lost without it also triggering a CRC error.

Originally Posted by Unity

If anyone disagrees with this conclusion, perhaps you could provide some citations on this? I would love to read up on it to learn why my understanding is wrong!

Chris

The NMEA paper is pretty good. Beyond that is from discussions with engineers building products, so no references, which I know doesn't help.