In what cases is SignalK needed?

[Tehani]

Originally Posted by nohal

What implementation that is? Looking at https://github.com/markert/ArduinoJsonParser it seems it's memory requrements are 1-2 orders of magnitude lower.

That implementation seems reasonable, but I'll give an example of how a simple Coordinate packet is transmitted in NMEA2000, NMEA0183 and SignalK:

NMEA2000:

0x09F80102,0x8,0x53 0xE2 0xC0 0x18 0x4E 0x68 0x75 0x01,0x4C69,ACK

Everything in pure binary: 29-bit identifier including priority, PGN and message sender. 4-bit DLC, 8 bytes of data, check and ACK. Rounding: 16 bytes

NMEA0183:

$GPGLL,5050.980,N,00120.055,W,110311.93,A,D*75

All ASCII, total: 48 bytes if we include CR+LF

Seatak 1

158 005 024 0A3 09F 006 061 048

8 characters of 9 bits in total: 1 char for the identifier, 2 bits for N/S E/W, 4 bits to indicate the data length and 6 data chars.

SignalK:

{
  "vessels": {
    "urn:mrn:signalk:uuid:705f5f1a-efaf-44aa-9cb8-a0fd6305567c": {
      "navigation": {
        "position": {
          "value": {
            "altitude": 0.0,
            "latitude": 37.81479,
            "longitude": -122.44880152
          },
          "source": {
            "label": "ttyUSB0",
            "type": "NMEA0183",
            "talker": "GP",
            "sentence": "PRMC"
          },
          "timestamp": "2017-05-16T05:15:50.007Z"
        }
      }
    }
  }
}

This in "Delta" or compressed. All ASCII, total message length: 524 bytes, and if we read, there are 113 bytes before we see any significant data.

Is this reasonable?

I woud be interested in understanding how you plan to fit the OneNet encryption and PKI based authentication into the same uC.

I see that you have not understood me, it is possible because of my difficulty with the language. In the environment where I am working, which is the communication of on-board devices, this does not worry me at all because it is a task that would be reserved for a server on the other side (based on TCP/UDP protocols...). I'm simply saying that the encapsulation, management and security would be in another layer in all cases, even SignalK.

J.L.

[nohal]

I still do not completely understand what your problem actually is.

You are kind of comparing apples with oranges - strip the SignalK message to contain the same amount of information as your other examples and the size will become much more similar (Of course still bigger when presented in the verbose JSON format).

If your problem really is the transmission layer throughput, nothing can stop you from using for example FlatBuffers there and packing it to a size efficient binary message, while still staying compatible with the SignalK object schema.

[Tehani]

Ok, it can also be sent like this:

{"vessels":{"urn:mrn:signalk:uuid:705f5f1a-efaf-44aa-9cb8-a0fd6305567c":{"navigation":{"position":{"value":{"altitude":0.0,"latitude":37.81479,"longitude":-122.44880152},"source":{"label":"ttyUSB0","type":"NMEA0183","talker":"GP","sentence":"PRMC"},"timestamp":"2017-05-16T05:15:50.007Z"}}}}}

Success: "only" 292 ASCII characters!!!

[nohal]

You are far from done and you already saved 50%, very good.

[Tehani]

Originally Posted by nohal

If your problem really is the transmission layer throughput, nothing can stop you from using for example FlatBuffers there and packing it to a size efficient binary message, while still staying compatible with the SignalK object schema.

FlatBuffers are used as standard in SignalK?. Everyone understands that without messing with more layers of code?

[Tehani]

Originally Posted by nohal

You are far from done and you already saved 50%, very good.

Good, because we understand each other.

[Tehani]

But it's too much for my limitations:

https://github.com/google/flatbuffers/tree/master/src

It may be nice, but I don't think I'm going to spend time on this. If something fails, I would go crazy.

[nohal]

Originally Posted by Tehani

FlatBuffers are used as standard in SignalK?. Everyone understands that without messing with more layers of code?

It seems like you might be confusing the SignalK schema with the reference implementation. The schema is completely usable independently from the encoding and transport used to transfer the data as far as I can tell.

And when we started nitpicking around the FlatBuffers example solution to some of your major complaints about SignalK, conversion to/from JSON seems not to be exactly rocket science as you can see for example in https://stackoverflow.com/questions/...s-to-from-json

[nohal]

Originally Posted by Tehani

But it's too much for my limitations:

https://github.com/google/flatbuffers/tree/master/src

It may be nice, but I don't think I'm going to spend time on this. If something fails, I would go crazy.

FWIW, FlatBuffers are routinely being used on STM32 microcontrollers, which AFAICT have similar specs as the device you talk about.
Of course, nobody forbids you from using whatever you consider suiting your needs, just please try to accept other people have different sets of requirements and constrainst and can likely take different approaches to address them than you would.

[Tehani]

Please can you tell me if OpenCPN (or any plugin) can currently receive and send FWIW FlatBuffers over a serial channel or WiFi?

I try to bring positions closer and agree on something to define an Open bidirectional communication between elementary devices and O. That is my goal.

However, and while that happens, I still approve the use of NMEA0183 for these things, but with some revision to update especially XDR and RPM according to the standard. I see that this is requested by users and that is what is missing.

J.L.

[Lake-Effect]

Originally Posted by Tehani

Please, at this point I think it is necessary to rigorously explain when it is essential for an OpenCPN user to use SignalK, because there are already many people stuck.

"Can be used" is not enough, because as far as OpenCPN is concerned, a conventional gateway is enough in the cases I know of: NMEA2000 <-> NMEA0183, and Seatalk <-> NMEA0183. And in any case, to use SignalK it is also necessary to use an external hardware device (We save nothing).

I don't know the precise answer, so I wish someone expert would kindly explain it in simple words.

I should point out that it is not my intention to go against SignalK, on the contrary, I think these guys are doing excellent documentation on NMEA2000.

Thanks
Jose Luis

Hello Jose, It seems you're developing something to work in an OpenCPN system? If yes, is not having SignalK compatibility going to hold you back?

Compared to XML, JSON is lightweight while still human-readable. I've used ArduinoJSON, mostly with success. And there are so many cheap MCUs and SBCs around that are capable of easily handling JSON at network speeds... I'd say that the main holdup to adding SignalK to a new device is the additional programming and testing time required, and with libraries, that's not huge.

[nohal]

Originally Posted by Tehani

Please can you tell me if OpenCPN (or any plugin) can currently receive and send FWIW FlatBuffers over a serial channel or WiFi?

Why should it, is there any counterpart for such communication? If there is, and uses FlatBuffers over TCP/IP or serial transport for some well defined message payload (for example SignalK), I will gladly add this support as the implementation would be trivial.

I have an inevitable feeling that the "problem" we have seems to be that we still did not settle even on the simple fact that FlatBuffers and similar are OSI layer 6 stuff, the technology used to encode messages, any messages, nothing else. What we need to be able to communicate is something at OSI layer 7 we all understand (that is what SignalK or NMEAxxxx or SeaTalk message format, not transport over which it is delivered is).

I try to bring positions closer and agree on something to define an Open bidirectional communication between elementary devices and O. That is my goal.

However, and while that happens, I still approve the use of NMEA0183 for these things, but with some revision to update especially XDR and RPM according to the standard. I see that this is requested by users and that is what is missing.

SignalK is the format that OpenCPN does already support, is well defined, open and readily available for the task.
NMEA0183 has no future, but if you feel OpenCPN's implementation of XDR and RPM is not standard compliant and needs fixing, please open an issue describing the exact problem and, if possible, suggested solution.

[250224]

Originally Posted by bdbcat

Jose...
From my perspective, the answer goes something like this:

1. NMEA 0183 is difficult to expand, and not network-friendly. Currently suffering from "add-on-itis".
2. NMEA2K is closed source, full of vendor-specific secret sauce.
3. SignalK is attempting, as one goal, to promulgate an open-source alternative to NMEA2K.
4. By implementing in sK OCPN, we help that process, by enabling real-world use of the code. We are materially helping to drive out inevitable bugs by seeing use on a variety of configurations. This we consider as our "contribution" to the sK effort.

Who needs sK on a reasonably complex OCPN installation? No one. But we (devs) sure appreciate those who do, and report results (and problems).

My 2 cents.
Dave

I see it less rosy (rather negative, I'm afraid):

* SignalK can't ever be an open-source alternative to NMEA2000(R). It's a distraction and serves NMEA well as they can point to this open protocol but don't need to fear anything at all
- This "standard built on common web technologies" is bloated; maybe ok to gather some values and display them in a browser and/or apps, but useless for low-level sensors or actuators. When looking at https://signalk.org/specification/1.5.0/doc/ and https://signalk.org/specification/1....ata_model.html I have major difficulties to see a clear definition, it's just a wall of json text (which some years ago I parsed and made a table-like schema (but then didn't want to engage))
- I think OCPN is weaker and not stronger with SignalK support, sk is a dead-end

* NMEA2000(R), or rather the PGNs may be proprietary but it's where the action is (and also will be in future). A good approach imho would be: EMBRACE and EXTEND. Like with SMB/SAMBA.
- 1) embrace the proprietary PGNs. This has and can be done with CanBoat. It would be nice to have a database with the most important PGNs in order to forego the dinosaur-like https://www.nmea.org/content/STANDAR...00_pgn__search site
- 2) extend. I.E. a group of enthusiasts could manage a set of free PGNs which could be registered in addition to the closed/proprietary PGNs. This of course would no longer be NMEA2000(R) but would be compatible and the Canbus network could just run with this open version

(I don't know enough, but I feel something like the TwoCan plugin should become (more) central in OCPN in future. Almost all (mainstream) boaters I know use n2k plotters/sensors or are thinking about upgrading to n2k)
(Last but not least: nothing against sk, it's a nice project, I just don't subscribe to the 'open-source alternative to NMEA2K' part)

[Tehani]

Originally Posted by Lake-Effect

Hello Jose, It seems you're developing something to work in an OpenCPN system? If yes, is not having SignalK compatibility going to hold you back?
...

Yes, I am finishing this project (See images):

I think a good part can be Open. At the moment, the hardware already is, because I already published it in this forum. Since I've made some corrections, I'll post it again.
It has a lot of legacy software from previous products, so I've been able to adapt it without much trouble.

At the moment it has the following connectivity (Everything is bidirectional):
- NMEA2000, 2x RS422 NMEA0183 and Seatalk 1
- USB, Wi-Fi
- Interface for 4-channel garage door receiver for autopilot control (seen in the photo).
The list of translated sentences and PGNs is very extensive, includes AIS and ARPA/MARPA messages, and everything is working now.
The compatibility with OpenCPN in NMEA0183 is total.

I am open to any proposal. I'm wondering now whether to add SignalK or not, but I don't have much memory anymore.
It is not urgent, because the device can be updated from a web browser.

[emilecantin]

@Tehani

In a world where you can have a ESP8266 for less than $2USD shipped, which can actually host a web interface over wifi, I think optimizing for RAM usage is a waste of time. If your MCU can't support JSON, get another one. You can get ESP32s with more memory for very cheap too (less than 10$).

And I'm happy to inform you that MCUs running Javascript actually do exist: https://shop.espruino.com/

SignalK isn't optimized for your use-case (hardcore embedded low-power usage), it's optimized for ease of developing other software around it. I can just take a look at the raw JSON and I immediately know where I can read the boat's position, no need to constantly refer to the documentation (and even if I do, it's free). N2K / CAN or N0183 is basically unreadable garbage if you try to debug it.

If you really need to lower your memory usage, there's nothing keeping you from talking to the SK server using a different protocol. It already understands NMEA0183, N2k and Seatalk1 natively, and can output these through a plugin. You can also create a plugin to support your own custom protocol if you want (I did this a while back when I started to mess around with SK, it was very easy).