Signal K Mobile App - Open Source

[teppokurki]

Originally Posted by boat_alexandra

Considering you need messages from gyros, accels, magnetic sensors and other sensors, either you need many ports, or you have to accept a huge amount of latency. The fusion filter needs to integrate all of these sensors at a a high rate (20-50hz minimum) to produce the most accurate results.

I totally agree that Signal K as JSON is not the tool for high bandwidth sensor data.

But take an ESP8266, attach a few temp sensors, connect to your wifi and start spitting out Signal K over UDP and you have a pretty nice addition to your boat's data. Then stop worrying about that alternator running hot because of the new regulator and create a custom screen for your smartphone, showing engine rpms and alternator temperature.

Originally Posted by boat_alexandra

It's basically unsuitable for:

1) serial port connections

Because of missing crc? Well, Signal K is open source and crc's aren't exactly rocket science, so there is nothing stopping us defining "SK over serial line" with crc. The message delimiter needs to be defined as well.

Originally Posted by boat_alexandra

2) low latency requirements -- wind direction, gyroscope etc..
3) high bandwidth -- radar, compute vision etc..

I agree.

Originally Posted by boat_alexandra

For serial ports (or usb serial ports), you can use a normal binary connection to the device, and a server program that translates this to signalk for the network which mostly solves it.

This is exactly how I see SK positioned: after data fusion, making human-oriented processed data in a standard format easily available to applications and application developers.

There is also a fundamental difference to NMEA0183/N2K: Signal K is extendable and evolvable. With NMEA0183 and N2K you are limited to the existing data structures - if you want to extend it, like use your own XDR sentences, then you are on your own. With Signal K you can propose an extension and have it included in the spec for everybody to share.

Originally Posted by boat_alexandra

I tried to explain to the signalk group that they didn't need to put the same identical timestamp 8 times in the same message and at minimal should support some kind of "ditto" but they didn't care they said just to get a faster wifi and faster processor so that it could handle this inefficient protocol. They are also java developers.

Timestamps are optional in delta format (the SK format meant for streaming data updates) and have been for quite a while: https://github.com/SignalK/specification/issues/92.

I think this summarises the situation pretty well: https://groups.google.com/d/msg/sign...Q/ZwqJt9EsCAAJ

As one of the primary authors of Signal K node server I don't know if I should take your "java developer" as a compliment or a rebuke, but it definitely is untrue in this context. But I guess JavaScript is even further down/up the ladder .

While there is quite a lot to improve in much of the code I put out an unqualified blanket statement about code quality is rather low imho.

Something that I'd like to point out is that Signal K is open source, in good and bad: it is what the community makes it. We have just introduced and are experimenting with an RFC process. Please join the discussion and let's make it better!

See https://groups.google.com/forum/#!to...lk/o_PJX8ke2Mw

But don't expect that the world famous Somebody Else implements all your ideas. An idea may be great, but if there is nobody writing the implementation and sharing it the idea will remain just words, albeit shared.

[teppokurki]

Originally Posted by Tenedos

I am in the process of building some DIY Signal K sensors for using on my own boat. Here is my somewhat prioritized list:

I may also need to develop the corresponding Signal K server code and then definitely the enhancement for my Signal K Mobile app to consume the input from those. So the net result is, it will all show up on the app somehow.

Pretty much the same that I did for my own boat's house battery current & voltage measurements: RPi, ADS1115 over I2C. See https://github.com/tkurki/cassiopeia...aster/index.js for the code that pulls the data from the ADC and pipes it to the Signal K server.

[seandepagnier]

thanks for the reply!

Originally Posted by teppokurki

I totally agree that Signal K as JSON is not the tool for high bandwidth sensor data.

But take an ESP8266, attach a few temp sensors, connect to your wifi and start spitting out Signal K over UDP and you have a pretty nice addition to your boat's data. Then stop worrying about that alternator running hot because of the new regulator and create a custom screen for your smartphone, showing engine rpms and alternator temperature.

I don't have a smartphone, engine, or any type of tank to measure.

Because of missing crc? Well, Signal K is open source and crc's aren't exactly rocket science, so there is nothing stopping us defining "SK over serial line" with crc. The message delimiter needs to be defined as well.

Sure, you can add a crc field. This might work in some cases.

This is exactly how I see SK positioned: after data fusion, making human-oriented processed data in a standard format easily available to applications and application developers.

I am not allowing any signalk anywhere between reading from the gyros and actually driving the motor to move the rudder.

The signalk accessible for feedback analysis control etc. It's slowness will not impact the autopilot performance.

There is also a fundamental difference to NMEA0183/N2K: Signal K is extendable and evolvable. With NMEA0183 and N2K you are limited to the existing data structures - if you want to extend it, like use your own XDR sentences, then you are on your own. With Signal K you can propose an extension and have it included in the spec for everybody to share.

Yes, but I have already implemented extensions that are non standard to signalk that I need, but the problem is everyone doing this creates something that is not so easy to deal with.

Timestamps are optional in delta format (the SK format meant for streaming data updates) and have been for quite a while: https://github.com/SignalK/specification/issues/92.

I am already treating all fields as options

I think this summarises the situation pretty well: https://groups.google.com/d/msg/sign...Q/ZwqJt9EsCAAJ

As one of the primary authors of Signal K node server I don't know if I should take your "java developer" as a compliment or a rebuke, but it

both would be reasonable. Did you write the java server implementation?

The fact that the only implementations are in java and javascript is a bit annoying as I have little interest in either language. I wrote one in python, but I know it doesn't "conform" to the standard exactly... I am trying to improve this somewhat, but I _need_ certain things that don't exist in signalk like clients registering to watch or modify certain values etc...

I need simple reflection like, enumerating all the possible values the server has and what types and ranges they have etc..

In any case the code is trivial so it can be reformulated.

definitely is untrue in this context. But I guess JavaScript is even further down/up the ladder .

Yes, javascript is for websites right? Why would I use it for a server?

While there is quite a lot to improve in much of the code I put out an unqualified blanket statement about code quality is rather low imho.

Good, I half did this to inspire a response... The code itself the java one is horrible to navigate because it uses 10 levels of directories???


I don't think the server should have any specifics that I am finding in the javacode for example, the java server is more than 3000 lines.

Also it is riddled with "freeboard" specific stuff which I have no idea why it's in the core server as few people will ever have this hardware.

At least it's using the correct software license.

By comparison my python version which is more flexible supports reflection etc, it is only 300 lines.

[Paul L]

Originally Posted by boat_alexandra

I ......... It's basically unsuitable for:

1) serial port connections
2) low latency requirements -- wind direction, gyroscope etc..
3) high bandwidth -- radar, compute vision etc..

........c

I can',t see how SK meets it's design goals as an extensible format for future communications when it seems too verbose and inefficient to handle current latency needs.

[teppokurki]

Originally Posted by Paul L

I can't see how SK meets its design goals as an extensible format for future communications when it seems too verbose and inefficient to handle current latency needs.

By being an open source project.

The current Signal K JSON over HTTP/WebSocket is not the right tool for low latency high bandwidth applications. It was never designed for that.

So what should we do? Let's devise a way to use the common Signal K data model for this!

Hmm let's pick an existing technology fit for this purpose, like Protobuf. Then let's define the way Signal K abstract data model keys and values (mostly floats and ints, some enums, some structs) are used with Protobuf. File an RFC, let's discuss & revise, then somebody writes a reference implementation, the RFC makes it to the Signal K spec and ... we have a way to transmit marine data in high bandwidth applications - with a shared vocabulary for mapping the data to different formats and purposes.

(Please don't pick on Protobuf here, I picked it as an example. I claim absolutely no expertise in this area.)
(Not everything needs to be Signal K for that matter. I don't see how or why radar data would fit in for example.)


You can go on writing your own stuff and publicly wonder "why Signal K does not do X".

Or you can help make it better.

This summer I needed the Signal K node server to do stuff that it could not do. I implemented the extensions that I needed in my own fork, quickly adding just what I needed.

But I have firm plans to share those extensions and at the same time add a generic extension mechanism to the server.

[teppokurki]

Originally Posted by boat_alexandra

I don't have a smartphone

Sorry about that, I assumed you were interested in Signal K Mobile apps, the title of this thread.

Originally Posted by boat_alexandra

I am trying to improve this somewhat, but I _need_ certain things that don't exist in signalk like clients registering to watch or modify certain values etc...

I need simple reflection like, enumerating all the possible values the server has and what types and ranges they have etc..

I would be most interested to hear about the extensions you have created, preferably on the Signal K mailing list or as concrete improvement/extension ideas at https://github.com/SignalK/specification/issues

Btw the Signal K spec has provisions for enum values and zones (labelled ranges) for values. I wonder if they might fit your needs. There is also the subscriptions protocol, which might do what you want re: registering to watch some values.

Originally Posted by boat_alexandra

The code itself the java one is horrible to navigate because it uses 10 levels of directories???

This particular gripe you need to take up with Oracle (nee Sun), since that is the way java code is organised.

Originally Posted by boat_alexandra

I don't think the server should have any specifics that I am finding in the javacode for example, the java server is more than 3000 lines.

Java server is Rob's freeboard server generously repurposed as a SK server.

If I remember correctly I have not contributed a single line to it personally.

[Tenedos]

Ok, here comes the first one: BME280 based temperature, pressure and humidity sensors. If you have a Raspberry Pi, this is just a $10 addition to that.

I wrote a small tutorial: Mobile App controlled mini weather station for Raspberry Pi.

It is essentially a combination of the mobile app that I announced on this thread and a daemon written in Python https://github.com/itemir/boatsensord.

It currently supports the BME280 sensor but one can easily create additional plugins to add more stuff. I intend to do that as we go.

Originally Posted by barnakiel

Off course!

Yesterday I was searching the web for Signal K sensors and found NONE.

So building protocols etc around non existent sensors is ... an exercise in vain?

I believe that much more exposure can be gained by open source'ing hardware like:

- depth, speed, positioning, wind, etc.

to talk with Signal K devices.

So. No doubt: if you are building any signal K sensors and if you want and can share the build, this will be pushing things forward.

Cheers,
b.

[Tenedos]

I also just did the same ;-) https://github.com/itemir/boatsensord

Would you mind sharing yours? I am trying to make sure this is Signal K compatible. We have an active discussion on the Signal K discussion list right now. Care to join?

Originally Posted by boat_alexandra

I wrote a signalk server in python for my own personal use and it is working.

[Tenedos]

Well, I am not a Javascript fan myself but this may not be totally fair either ;-) Server side javascript has been very popular lately in a lot of prominent projects. While I personally would have preferred Python too (as I mentioned in previous posts, I wrote my simplified Python version) I don't think there is any issue with Javascript. I also find it very hard to read Java but again, those are personal choices. It is the de-facto for others.

What I will say on the topic is more about the standard than the actual implementations. The current emphasis seems to be a bit more about the implementations and different pieces of code than the standard and use-cases it addresses in practice. While these all need to be balanced, I also think that a bit of better documentation and plug-in-play type solutions will help advancement of the protocol. For instance:

• What/if any sensors are supported out of the box? How can we make it so that more are supported out of the box without tinkering?
• Is there a way we can document this more crisply? I have been going back and forth in documentation, code and discussion lists and I am not sure I am understanding all the details. I still haven't figured out if it is just me, but I feel like some clearer (and maybe simpler) documentation would be helpful. Maybe with a couple of sample use cases.
• Existing code and the protocol itself are a bit verbose and there seems to be multiple ways and components for doing the same thing. Being an engineer I get the value of 'the more the better and flexible' approach but with a Product Manager hat on, the more is not always better. Could we make it a more concise and streamlined story with code examples, documentation and how-to style guides?

These are just thoughts. I do value open standards and I don't think they are there to advance by themselves or solve every problem on the planet. Bringing volunteer groups together and achieving things is not easy and needs to be respected. I think we can make this better together.

Originally Posted by boat_alexandra

Yes, javascript is for websites right? Why would I use it for a server?

Good, I half did this to inspire a response... The code itself the java one is horrible to navigate because it uses 10 levels of directories???


I don't think the server should have any specifics that I am finding in the javacode for example, the java server is more than 3000 lines.

Also it is riddled with "freeboard" specific stuff which I have no idea why it's in the core server as few people will ever have this hardware.

At least it's using the correct software license.

By comparison my python version which is more flexible supports reflection etc, it is only 300 lines.

[Tenedos]

Yes, OpenPlotter is cool. I think one of the use cases is to access data onboard, that is more the OpenPlotter domain.

But these days my boat is mostly at the marina when I am on and off. I am looking at a use case to access that data remotely. Sure it can be accessed from the database remotely with a VPN connection but that I think is not a very easy setup for many.

I think Signal K is good when you want to take the data off the boat and on the Internet.

Originally Posted by conachair

..
Have you checked out openplotter? V0.9 looks to be very powerful in signalk options. I set up rrdtools on the boat's pi as well so it records everything into a database once a minute. Just nmea at the moment but should be easy to parse signalk data as well.
...

Interesting times, and open source away from the nmea stranglehold.

[Tenedos]

Nice and funny that is one of the next two plugins in the pipe for my sensor project. I am planning to use the MCP3004/3008 for the ADC.

I get my voltage readings for the battery from the panel itself, how do you get the current readings?

Originally Posted by teppokurki

Pretty much the same that I did for my own boat's house battery current & voltage measurements: RPi, ADS1115 over I2C. See https://github.com/tkurki/cassiopeia...aster/index.js for the code that pulls the data from the ADC and pipes it to the Signal K server.

[barnakiel]

Originally Posted by Tenedos

I also just did the same ;-) https://github.com/itemir/boatsensord

Would you mind sharing yours? I am trying to make sure this is Signal K compatible. We have an active discussion on the Signal K discussion list right now. Care to join?

Great job!

Can the source be rewritten to output nmea?

And what is your opinion on deploying on e.g. NanoPi?

Smaller & Faster than Raspberry Pi Zero: Meet NanoPi NEO ARM Linux Development Board

Thanks & regards,
b.

[Tenedos]

Thank you.

I am not well versed in the NMEA protocol itself beyond being a user and I don't have NMEA 2000 on my boat. If it does support temperature, pressure and humidity readings, it should be doable. But then that is the problem of closed protocols and why I like Signal K ;-)

The code should work on NanoPi or any other platform that has I2C support that can integrate with the sensor. Not sure about the library compatibility though. I use the Adafruit GPIO library to access BME280, it may or may not be compatible with the GPIO layout of NanoPi.

One thing about all these platforms is the ecosystem. Having a broad ecosystem really does help for development as there are more people, more discussion, more testing and more mature libraries available. It significantly speeds up development!

Originally Posted by barnakiel

Great job!

Can the source be rewritten to output nmea?

And what is your opinion on deploying on e.g. NanoPi?

Smaller & Faster than Raspberry Pi Zero: Meet NanoPi NEO ARM Linux Development Board

Thanks & regards,
b.

[teppokurki]

Originally Posted by Tenedos

I get my voltage readings for the battery from the panel itself, how do you get the current readings?

With a simple voltage divider. Current is measured by ADS1115 as voltage across the battery panel's shunt.

[barnakiel]

Thank you again!

I build small custom things for our own boat or for the friends and only for fun. I do not write my code but rather pick up pieces from varied sources and combine them to get what works.

I am still trying to wrap my head round the Signal K thing. This is also where I differ from your opinion on rapid development. My take is that for projects like K support to the external world is nil to minimal - people in the circles have resolved to speak their own hermetic lingo and this practically cuts away input and output towards third parties. Ergo: such developers write beautiful pieces of code. But they end up very far from where their market is.

This reminds me very much "support" for Linuks users. 'Your question has been already answered', 'Your question is ill formed', 'Fire up the terminal and try ...'. ;-)

This does not push things forward. This creates an artificial ego barrier between the feedback guy / user and the coder.

One cannot rapidly develop useful hardware and software without rapid feedback from the end users. And such feedback is not possible without speaking a human language. How silly Signal K people talk about 'human readable' data flow meanwhile remaining themselves hardly human readable.

Having participated in big building teams that developed software for the banking industry I look at the open source world and I wonder. Why the adjective 'open'? It does not feel any more open than the things we built with proprietary software, for proprietary platforms. It is an oxymoron, like free beer and healthy fast food.

There must be an easier and better way to build open source hardware and software. Maybe it is yet to come.

Apologies for the rant.

Thanks again, I stay tuned.

barnakiel