Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Rate Thread Display Modes
Old 04-10-2016, 12:25   #16
Registered User

Join Date: Feb 2016
Posts: 16
Re: Signal K Mobile App - Open Source

Quote:
Originally Posted by boat_alexandra View Post
Considering you need messages from gyros, accels, magnetic sensors and other sensors, either you need many ports, or you have to accept a huge amount of latency. The fusion filter needs to integrate all of these sensors at a a high rate (20-50hz minimum) to produce the most accurate results.
I totally agree that Signal K as JSON is not the tool for high bandwidth sensor data.

But take an ESP8266, attach a few temp sensors, connect to your wifi and start spitting out Signal K over UDP and you have a pretty nice addition to your boat's data. Then stop worrying about that alternator running hot because of the new regulator and create a custom screen for your smartphone, showing engine rpms and alternator temperature.

Quote:
Originally Posted by boat_alexandra View Post
It's basically unsuitable for:

1) serial port connections
Because of missing crc? Well, Signal K is open source and crc's aren't exactly rocket science, so there is nothing stopping us defining "SK over serial line" with crc. The message delimiter needs to be defined as well.

Quote:
Originally Posted by boat_alexandra View Post
2) low latency requirements -- wind direction, gyroscope etc..
3) high bandwidth -- radar, compute vision etc..
I agree.

Quote:
Originally Posted by boat_alexandra View Post
For serial ports (or usb serial ports), you can use a normal binary connection to the device, and a server program that translates this to signalk for the network which mostly solves it.
This is exactly how I see SK positioned: after data fusion, making human-oriented processed data in a standard format easily available to applications and application developers.

There is also a fundamental difference to NMEA0183/N2K: Signal K is extendable and evolvable. With NMEA0183 and N2K you are limited to the existing data structures - if you want to extend it, like use your own XDR sentences, then you are on your own. With Signal K you can propose an extension and have it included in the spec for everybody to share.

Quote:
Originally Posted by boat_alexandra View Post
I tried to explain to the signalk group that they didn't need to put the same identical timestamp 8 times in the same message and at minimal should support some kind of "ditto" but they didn't care they said just to get a faster wifi and faster processor so that it could handle this inefficient protocol. They are also java developers.
Timestamps are optional in delta format (the SK format meant for streaming data updates) and have been for quite a while: https://github.com/SignalK/specification/issues/92.

I think this summarises the situation pretty well: https://groups.google.com/d/msg/sign...Q/ZwqJt9EsCAAJ

As one of the primary authors of Signal K node server I don't know if I should take your "java developer" as a compliment or a rebuke, but it definitely is untrue in this context. But I guess JavaScript is even further down/up the ladder .

While there is quite a lot to improve in much of the code I put out an unqualified blanket statement about code quality is rather low imho.

Something that I'd like to point out is that Signal K is open source, in good and bad: it is what the community makes it. We have just introduced and are experimenting with an RFC process. Please join the discussion and let's make it better!

See https://groups.google.com/forum/#!to...lk/o_PJX8ke2Mw

But don't expect that the world famous Somebody Else implements all your ideas. An idea may be great, but if there is nobody writing the implementation and sharing it the idea will remain just words, albeit shared.
__________________

__________________
teppokurki is offline   Reply With Quote
Old 04-10-2016, 12:31   #17
Registered User

Join Date: Feb 2016
Posts: 16
Re: Signal K Mobile App - Open Source

Quote:
Originally Posted by Tenedos View Post
I am in the process of building some DIY Signal K sensors for using on my own boat. Here is my somewhat prioritized list:

I may also need to develop the corresponding Signal K server code and then definitely the enhancement for my Signal K Mobile app to consume the input from those. So the net result is, it will all show up on the app somehow.
Pretty much the same that I did for my own boat's house battery current & voltage measurements: RPi, ADS1115 over I2C. See https://github.com/tkurki/cassiopeia...aster/index.js for the code that pulls the data from the ADC and pipes it to the Signal K server.
__________________

__________________
teppokurki is offline   Reply With Quote
Old 04-10-2016, 17:15   #18
Registered User
 
boat_alexandra's Avatar

Join Date: Aug 2009
Location: BVI
Boat: bristol 27
Posts: 2,676
Re: Signal K Mobile App - Open Source

thanks for the reply!

Quote:
Originally Posted by teppokurki View Post
I totally agree that Signal K as JSON is not the tool for high bandwidth sensor data.

But take an ESP8266, attach a few temp sensors, connect to your wifi and start spitting out Signal K over UDP and you have a pretty nice addition to your boat's data. Then stop worrying about that alternator running hot because of the new regulator and create a custom screen for your smartphone, showing engine rpms and alternator temperature.
I don't have a smartphone, engine, or any type of tank to measure.
Quote:
Because of missing crc? Well, Signal K is open source and crc's aren't exactly rocket science, so there is nothing stopping us defining "SK over serial line" with crc. The message delimiter needs to be defined as well.
Sure, you can add a crc field. This might work in some cases.
Quote:

This is exactly how I see SK positioned: after data fusion, making human-oriented processed data in a standard format easily available to applications and application developers.
I am not allowing any signalk anywhere between reading from the gyros and actually driving the motor to move the rudder.

The signalk accessible for feedback analysis control etc. It's slowness will not impact the autopilot performance.
Quote:
There is also a fundamental difference to NMEA0183/N2K: Signal K is extendable and evolvable. With NMEA0183 and N2K you are limited to the existing data structures - if you want to extend it, like use your own XDR sentences, then you are on your own. With Signal K you can propose an extension and have it included in the spec for everybody to share.
Yes, but I have already implemented extensions that are non standard to signalk that I need, but the problem is everyone doing this creates something that is not so easy to deal with.

Quote:
Timestamps are optional in delta format (the SK format meant for streaming data updates) and have been for quite a while: https://github.com/SignalK/specification/issues/92.
I am already treating all fields as options
Quote:
I think this summarises the situation pretty well: https://groups.google.com/d/msg/sign...Q/ZwqJt9EsCAAJ

As one of the primary authors of Signal K node server I don't know if I should take your "java developer" as a compliment or a rebuke, but it
both would be reasonable. Did you write the java server implementation?

The fact that the only implementations are in java and javascript is a bit annoying as I have little interest in either language. I wrote one in python, but I know it doesn't "conform" to the standard exactly... I am trying to improve this somewhat, but I _need_ certain things that don't exist in signalk like clients registering to watch or modify certain values etc...

I need simple reflection like, enumerating all the possible values the server has and what types and ranges they have etc..

In any case the code is trivial so it can be reformulated.
Quote:
definitely is untrue in this context. But I guess JavaScript is even further down/up the ladder .
Yes, javascript is for websites right? Why would I use it for a server?

Quote:
While there is quite a lot to improve in much of the code I put out an unqualified blanket statement about code quality is rather low imho.
Good, I half did this to inspire a response... The code itself the java one is horrible to navigate because it uses 10 levels of directories???


I don't think the server should have any specifics that I am finding in the javacode for example, the java server is more than 3000 lines.

Also it is riddled with "freeboard" specific stuff which I have no idea why it's in the core server as few people will ever have this hardware.

At least it's using the correct software license.

By comparison my python version which is more flexible supports reflection etc, it is only 300 lines.
__________________
boat_alexandra is offline   Reply With Quote
Old 04-10-2016, 17:36   #19
Senior Cruiser

Cruisers Forum Supporter

Join Date: Jul 2005
Boat: Outbound 44
Posts: 3,893
Re: Signal K Mobile App - Open Source

Quote:
Originally Posted by boat_alexandra View Post
I ......... It's basically unsuitable for:

1) serial port connections
2) low latency requirements -- wind direction, gyroscope etc..
3) high bandwidth -- radar, compute vision etc..

........c
I can',t see how SK meets it's design goals as an extensible format for future communications when it seems too verbose and inefficient to handle current latency needs.
__________________
Paul L
http://svjeorgia.blogspot.com
Paul L is offline   Reply With Quote
Old 05-10-2016, 03:24   #20
Registered User

Join Date: Feb 2016
Posts: 16
Re: Signal K Mobile App - Open Source

Quote:
Originally Posted by Paul L View Post
I can't see how SK meets its design goals as an extensible format for future communications when it seems too verbose and inefficient to handle current latency needs.
By being an open source project.

The current Signal K JSON over HTTP/WebSocket is not the right tool for low latency high bandwidth applications. It was never designed for that.

So what should we do? Let's devise a way to use the common Signal K data model for this!

Hmm let's pick an existing technology fit for this purpose, like Protobuf. Then let's define the way Signal K abstract data model keys and values (mostly floats and ints, some enums, some structs) are used with Protobuf. File an RFC, let's discuss & revise, then somebody writes a reference implementation, the RFC makes it to the Signal K spec and ... we have a way to transmit marine data in high bandwidth applications - with a shared vocabulary for mapping the data to different formats and purposes.

(Please don't pick on Protobuf here, I picked it as an example. I claim absolutely no expertise in this area.)
(Not everything needs to be Signal K for that matter. I don't see how or why radar data would fit in for example.)


You can go on writing your own stuff and publicly wonder "why Signal K does not do X".

Or you can help make it better.

This summer I needed the Signal K node server to do stuff that it could not do. I implemented the extensions that I needed in my own fork, quickly adding just what I needed.

But I have firm plans to share those extensions and at the same time add a generic extension mechanism to the server.
__________________
teppokurki is offline   Reply With Quote
Old 05-10-2016, 06:25   #21
Registered User

Join Date: Feb 2016
Posts: 16
Re: Signal K Mobile App - Open Source

Quote:
Originally Posted by boat_alexandra View Post
I don't have a smartphone
Sorry about that, I assumed you were interested in Signal K Mobile apps, the title of this thread.

Quote:
Originally Posted by boat_alexandra View Post
I am trying to improve this somewhat, but I _need_ certain things that don't exist in signalk like clients registering to watch or modify certain values etc...

I need simple reflection like, enumerating all the possible values the server has and what types and ranges they have etc..
I would be most interested to hear about the extensions you have created, preferably on the Signal K mailing list or as concrete improvement/extension ideas at https://github.com/SignalK/specification/issues

Btw the Signal K spec has provisions for enum values and zones (labelled ranges) for values. I wonder if they might fit your needs. There is also the subscriptions protocol, which might do what you want re: registering to watch some values.

Quote:
Originally Posted by boat_alexandra View Post
The code itself the java one is horrible to navigate because it uses 10 levels of directories???
This particular gripe you need to take up with Oracle (nee Sun), since that is the way java code is organised.

Quote:
Originally Posted by boat_alexandra View Post
I don't think the server should have any specifics that I am finding in the javacode for example, the java server is more than 3000 lines.
Java server is Rob's freeboard server generously repurposed as a SK server.

If I remember correctly I have not contributed a single line to it personally.
__________________
teppokurki is offline   Reply With Quote
Old 05-10-2016, 09:30   #22
Registered User

Join Date: Nov 2012
Location: San Francisco Bay
Boat: 1999 Catalina 320
Posts: 105
Re: Signal K Mobile App - Open Source

Ok, here comes the first one: BME280 based temperature, pressure and humidity sensors. If you have a Raspberry Pi, this is just a $10 addition to that.

I wrote a small tutorial: Mobile App controlled mini weather station for Raspberry Pi.

It is essentially a combination of the mobile app that I announced on this thread and a daemon written in Python https://github.com/itemir/boatsensord.

It currently supports the BME280 sensor but one can easily create additional plugins to add more stuff. I intend to do that as we go.

Quote:
Originally Posted by barnakiel View Post
Off course!

Yesterday I was searching the web for Signal K sensors and found NONE.

So building protocols etc around non existent sensors is ... an exercise in vain?

I believe that much more exposure can be gained by open source'ing hardware like:

- depth, speed, positioning, wind, etc.

to talk with Signal K devices.

So. No doubt: if you are building any signal K sensors and if you want and can share the build, this will be pushing things forward.

Cheers,
b.
__________________
Tenedos is offline   Reply With Quote
Old 05-10-2016, 09:33   #23
Registered User

Join Date: Nov 2012
Location: San Francisco Bay
Boat: 1999 Catalina 320
Posts: 105
Re: Signal K Mobile App - Open Source

I also just did the same ;-) https://github.com/itemir/boatsensord

Would you mind sharing yours? I am trying to make sure this is Signal K compatible. We have an active discussion on the Signal K discussion list right now. Care to join?

Quote:
Originally Posted by boat_alexandra View Post
I wrote a signalk server in python for my own personal use and it is working.
__________________
Tenedos is offline   Reply With Quote
Old 05-10-2016, 09:50   #24
Registered User

Join Date: Nov 2012
Location: San Francisco Bay
Boat: 1999 Catalina 320
Posts: 105
Re: Signal K Mobile App - Open Source

Well, I am not a Javascript fan myself but this may not be totally fair either ;-) Server side javascript has been very popular lately in a lot of prominent projects. While I personally would have preferred Python too (as I mentioned in previous posts, I wrote my simplified Python version) I don't think there is any issue with Javascript. I also find it very hard to read Java but again, those are personal choices. It is the de-facto for others.

What I will say on the topic is more about the standard than the actual implementations. The current emphasis seems to be a bit more about the implementations and different pieces of code than the standard and use-cases it addresses in practice. While these all need to be balanced, I also think that a bit of better documentation and plug-in-play type solutions will help advancement of the protocol. For instance:
  • What/if any sensors are supported out of the box? How can we make it so that more are supported out of the box without tinkering?
  • Is there a way we can document this more crisply? I have been going back and forth in documentation, code and discussion lists and I am not sure I am understanding all the details. I still haven't figured out if it is just me, but I feel like some clearer (and maybe simpler) documentation would be helpful. Maybe with a couple of sample use cases.
  • Existing code and the protocol itself are a bit verbose and there seems to be multiple ways and components for doing the same thing. Being an engineer I get the value of 'the more the better and flexible' approach but with a Product Manager hat on, the more is not always better. Could we make it a more concise and streamlined story with code examples, documentation and how-to style guides?

These are just thoughts. I do value open standards and I don't think they are there to advance by themselves or solve every problem on the planet. Bringing volunteer groups together and achieving things is not easy and needs to be respected. I think we can make this better together.

Quote:
Originally Posted by boat_alexandra View Post
Yes, javascript is for websites right? Why would I use it for a server?

Good, I half did this to inspire a response... The code itself the java one is horrible to navigate because it uses 10 levels of directories???


I don't think the server should have any specifics that I am finding in the javacode for example, the java server is more than 3000 lines.

Also it is riddled with "freeboard" specific stuff which I have no idea why it's in the core server as few people will ever have this hardware.

At least it's using the correct software license.

By comparison my python version which is more flexible supports reflection etc, it is only 300 lines.
__________________
Tenedos is offline   Reply With Quote
Old 05-10-2016, 09:57   #25
Registered User

Join Date: Nov 2012
Location: San Francisco Bay
Boat: 1999 Catalina 320
Posts: 105
Re: Signal K Mobile App - Open Source

Yes, OpenPlotter is cool. I think one of the use cases is to access data onboard, that is more the OpenPlotter domain.

But these days my boat is mostly at the marina when I am on and off. I am looking at a use case to access that data remotely. Sure it can be accessed from the database remotely with a VPN connection but that I think is not a very easy setup for many.

I think Signal K is good when you want to take the data off the boat and on the Internet.

Quote:
Originally Posted by conachair View Post
..
Have you checked out openplotter? V0.9 looks to be very powerful in signalk options. I set up rrdtools on the boat's pi as well so it records everything into a database once a minute. Just nmea at the moment but should be easy to parse signalk data as well.
...

Interesting times, and open source away from the nmea stranglehold.
__________________
Tenedos is offline   Reply With Quote
Old 05-10-2016, 10:49   #26
Registered User

Join Date: Nov 2012
Location: San Francisco Bay
Boat: 1999 Catalina 320
Posts: 105
Re: Signal K Mobile App - Open Source

Nice and funny that is one of the next two plugins in the pipe for my sensor project. I am planning to use the MCP3004/3008 for the ADC.

I get my voltage readings for the battery from the panel itself, how do you get the current readings?

Quote:
Originally Posted by teppokurki View Post
Pretty much the same that I did for my own boat's house battery current & voltage measurements: RPi, ADS1115 over I2C. See https://github.com/tkurki/cassiopeia...aster/index.js for the code that pulls the data from the ADC and pipes it to the Signal K server.
__________________
Tenedos is offline   Reply With Quote
Old 05-10-2016, 16:13   #27
Senior Cruiser

Cruisers Forum Supporter

Join Date: Aug 2009
Location: between the devil and the deep blue sea
Boat: a sailing boat
Posts: 16,589
Re: Signal K Mobile App - Open Source

Quote:
Originally Posted by Tenedos View Post
I also just did the same ;-) https://github.com/itemir/boatsensord

Would you mind sharing yours? I am trying to make sure this is Signal K compatible. We have an active discussion on the Signal K discussion list right now. Care to join?
Great job!

Can the source be rewritten to output nmea?

And what is your opinion on deploying on e.g. NanoPi?

Smaller & Faster than Raspberry Pi Zero: Meet NanoPi NEO ARM Linux Development Board

Thanks & regards,
b.
__________________
barnakiel is offline   Reply With Quote
Old 05-10-2016, 17:20   #28
Registered User

Join Date: Nov 2012
Location: San Francisco Bay
Boat: 1999 Catalina 320
Posts: 105
Re: Signal K Mobile App - Open Source

Thank you.

I am not well versed in the NMEA protocol itself beyond being a user and I don't have NMEA 2000 on my boat. If it does support temperature, pressure and humidity readings, it should be doable. But then that is the problem of closed protocols and why I like Signal K ;-)

The code should work on NanoPi or any other platform that has I2C support that can integrate with the sensor. Not sure about the library compatibility though. I use the Adafruit GPIO library to access BME280, it may or may not be compatible with the GPIO layout of NanoPi.

One thing about all these platforms is the ecosystem. Having a broad ecosystem really does help for development as there are more people, more discussion, more testing and more mature libraries available. It significantly speeds up development!

Quote:
Originally Posted by barnakiel View Post
Great job!

Can the source be rewritten to output nmea?

And what is your opinion on deploying on e.g. NanoPi?

Smaller & Faster than Raspberry Pi Zero: Meet NanoPi NEO ARM Linux Development Board

Thanks & regards,
b.
__________________
Tenedos is offline   Reply With Quote
Old 05-10-2016, 23:16   #29
Registered User

Join Date: Feb 2016
Posts: 16
Re: Signal K Mobile App - Open Source

Quote:
Originally Posted by Tenedos View Post
I get my voltage readings for the battery from the panel itself, how do you get the current readings?
With a simple voltage divider. Current is measured by ADS1115 as voltage across the battery panel's shunt.
__________________
teppokurki is offline   Reply With Quote
Old 06-10-2016, 07:14   #30
Senior Cruiser

Cruisers Forum Supporter

Join Date: Aug 2009
Location: between the devil and the deep blue sea
Boat: a sailing boat
Posts: 16,589
Re: Signal K Mobile App - Open Source

Thank you again!

I build small custom things for our own boat or for the friends and only for fun. I do not write my code but rather pick up pieces from varied sources and combine them to get what works.

I am still trying to wrap my head round the Signal K thing. This is also where I differ from your opinion on rapid development. My take is that for projects like K support to the external world is nil to minimal - people in the circles have resolved to speak their own hermetic lingo and this practically cuts away input and output towards third parties. Ergo: such developers write beautiful pieces of code. But they end up very far from where their market is.

This reminds me very much "support" for Linuks users. 'Your question has been already answered', 'Your question is ill formed', 'Fire up the terminal and try ...'. ;-)

This does not push things forward. This creates an artificial ego barrier between the feedback guy / user and the coder.

One cannot rapidly develop useful hardware and software without rapid feedback from the end users. And such feedback is not possible without speaking a human language. How silly Signal K people talk about 'human readable' data flow meanwhile remaining themselves hardly human readable.

Having participated in big building teams that developed software for the banking industry I look at the open source world and I wonder. Why the adjective 'open'? It does not feel any more open than the things we built with proprietary software, for proprietary platforms. It is an oxymoron, like free beer and healthy fast food.

There must be an easier and better way to build open source hardware and software. Maybe it is yet to come.

Apologies for the rant.

Thanks again, I stay tuned.

barnakiel
__________________

__________________
barnakiel is offline   Reply With Quote
Reply

Tags
mob

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
New Mobile browsing option | Mobile skin Janet H Forum News & Announcements 7 03-03-2015 18:33
Signal K open source universal marine data exchange rgleason OpenCPN 0 16-11-2014 08:17
Aqua Signal Series 43 How to Open, Bulb Replacement DavefromNZ Construction, Maintenance & Refit 0 06-03-2013 17:22
Which Navigation App Do I Choose for My Mobile Device ? r.fairman Navigation 0 18-11-2011 23:14


Our Communities

Our communities encompass many different hobbies and interests, but each one is built on friendly, intelligent membership.

» More about our Communities

Automotive Communities

Our Automotive communities encompass many different makes and models. From U.S. domestics to European Saloons.

» More about our Automotive Communities

Marine Communities

Our Marine websites focus on Cruising and Sailing Vessels, including forums and the largest cruising Wiki project on the web today.

» More about our Marine Communities


Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 02:23.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.