Cruisers Forum
 


Reply
  This discussion is proudly sponsored by:
Please support our sponsors and let them know you heard about their products on Cruisers Forums. Advertise Here
 
Thread Tools Search this Thread Rate Thread Display Modes
Old 05-06-2006, 20:54   #1
Registered User

Join Date: Feb 2006
Location: Portland, Maine
Posts: 24
Send a message via AIM to MaineCub Send a message via Yahoo to MaineCub
Open Source Navigation

Ookay... First off, I would hardly call myself a sailor. It is a dream of mine. With that said, what I do know (fairly well) is technology.

I as a application developer (computer programmer) have access to a multitude of programs out there (through the internet) under a category of "Open Source." In laymans terms, it essentially means free to use, update, change and redestribute (mostly for non commerical applications). Most of these open source programs are created, managed and maintained by a dedicated group of individuals. Also as a sociologist (that's my academic background), I believe it is in response to the Microsoft's of the world. Evening out the playing field so to speak.

I am surprised that a concept like this hasn't taken place in the boating/navigation industry. I am by no means an engineer.. But I don't see how difficult it would be to 'tie' a GPS to a depth finder and a recorder of some sorts. The recorder could then on a timely basis report it's data back to a central location. Subscribers could download the data into their navigation machines etc and use.

It seems very simple to me? Does something like this already exist? If not, lets get started!!!

Best,

--Dave
Portland, Maine
MaineCub is offline   Reply With Quote
Old 06-06-2006, 02:35   #2
Senior Cruiser
 
GordMay's Avatar

Cruisers Forum Supporter

Join Date: Mar 2003
Location: Thunder Bay, Ontario - 48-29N x 89-20W
Boat: (Cruiser Living On Dirt)
Posts: 49,082
Images: 241
”... lets get started!!!” is a great idea.
In this case, the “us” in “let’s” being more accurately defined as “you”.
__________________
Gord May
"If you didn't have the time or money to do it right in the first place, when will you get the time/$ to fix it?"



GordMay is offline   Reply With Quote
Old 06-06-2006, 06:00   #3
Registered User

Join Date: Feb 2006
Location: Portland, Maine
Posts: 24
Send a message via AIM to MaineCub Send a message via Yahoo to MaineCub
Well.. I've done some research. It looks like the major component of this type of application has already been created

http://javagps.sourceforge.net/

It's piqued my interest..
MaineCub is offline   Reply With Quote
Old 06-06-2006, 12:21   #4
Registered User
 
DeepFrz's Avatar

Cruisers Forum Supporter

Join Date: Mar 2006
Location: Winnipeg
Boat: None at this time
Posts: 8,462
Dave, hi, glad to hear someone is interested in this. I did some work several years ago but gave up. Mostly reasearching the ECS specifications. I couldn't afford the complete kit from IHO so I worked from bits and pieces that I downloaded.

I wasn't aware of the Java software available on sourceforge. There is also some C++ code around but it's been a long time since I programmed in C++.

I would be interested in helping out an open source project. I did some work on ArgoUML a few years back. It will take me a while to get back up to speed as I haven't done any programming in the last 2 or 3 years.

Keep me informed of your plans. In the meantime I will take a look at JavaGPS.

Deep ( address is philsa at mts dot net )
DeepFrz is offline   Reply With Quote
Old 06-06-2006, 14:45   #5
Registered User

Join Date: Feb 2006
Location: Portland, Maine
Posts: 24
Send a message via AIM to MaineCub Send a message via Yahoo to MaineCub
I think what I need to do is blow the dust off a computer I got laying around, install Fedora on it, Subversion (version control system), and see where it takes me.

I by no means am the greatest coder, I suffer from the "my eyes are bigger than my stomach" syndrome. But it's an idea I've had in my head for bout 6 months now.

First step is determining how NMEA devices work, NMEA and USB I suppose. NMEA Out ports? If you had two devices: one a GPS device, and another a depth finder or one device with both GPS and Depth? (sorry, I know very little about the strucutre of marine electronics devices)

Think my second step is learning how Java interacts with NMEA devices. Shoudln't be too difficult.

Lets see where this goes..

Lets keep in touch,

--Dave
MaineCub is offline   Reply With Quote
Old 06-06-2006, 15:13   #6
Registered User
 
DeepFrz's Avatar

Cruisers Forum Supporter

Join Date: Mar 2006
Location: Winnipeg
Boat: None at this time
Posts: 8,462
Dave,
I use Debian and eclipse IDE. eclipse has a "team" feature that I have never used, esentially VCS I think.

Phil.
DeepFrz is offline   Reply With Quote
Old 06-06-2006, 19:01   #7
Registered User

Join Date: Feb 2006
Location: Portland, Maine
Posts: 24
Send a message via AIM to MaineCub Send a message via Yahoo to MaineCub
Quote:
Originally Posted by DeepFrz
Dave,
I use Debian and eclipse IDE. eclipse has a "team" feature that I have never used, esentially VCS I think.

Phil.
I think what you are referring to is CVS. Subversion (Subclipse) has essentially replaced CVS. To install:

http://subclipse.tigris.org/install.html

Like I said, Im setting up Fedora (Linux) server at my house. I'll set up a instance of subversion on this server. From there we can write code and submit it to the versioning system (like they do at Sourceforge)

I go the new puter set up, but now need to install Fedora on it. That'll be tomorrow project!

Best,

--Dave
MaineCub is offline   Reply With Quote
Old 06-06-2006, 20:42   #8
֍֎֍֎֍֎֍֎֍֎

Join Date: Apr 2006
Posts: 15,136
Dave, if you mean having sailors gather position and depth information and forward that into a central mapping agency? Yes, that's how things are/were done. Sailors were openly invited to send data into the Defense Hydrogrpahic and Mapping Agency (DHMA) which since has been absorbed into NIMA or whatever they are calling it this week. There was no provision for digital data handling because mapping was done manually not so long ago.
But now? There's so much data available, I think they would rather not open it up to unverified data from unverified sources just to clutter in the thin parts of the maps, where commercial traffic isn't often found.
You could of course contact them to find out what their current position is, as The Keepers of The Data.
hellosailor is offline   Reply With Quote
Old 06-06-2006, 22:37   #9
Registered User
 
coot's Avatar

Join Date: Feb 2006
Posts: 367
Images: 2
Before you go installing software and selecting development tools, you might spend a little bit of time on Requirements Analysis and Software Design.

If you gather data from many sources with an open-source program, a few things come to mind:
- What is your data exchange format?
- How will you deal with GPS accuracy? (i.e. HDOP - horizontal dilution of position)
- How will you correct for tide variations? Not just high vs low tide, but the water level is higher because the wind is from the south for the last 2 days.
- How will you prevent malicious users from uploading false data into your system?

There is a joke that you can write any program by starting with the program "x=0;" and debugging it. It's a funny joke because so many people really do seem to write their programs that way, and that is part of why so much software is such crap. It is very helpful to re-use somebody else's existing software, but only if there is a pretty good correspondence between what it does and what you want to do.

In this case, the principal operation is to integrate data from multiple sources into a single data store. That is the hard part. Collecting NMEA data is trivial. If you don't concentrate too closely on NMEA format, you could also use Seatalk data. (Do a web search to find a guy who reverse engineered parts of the Seatalk protocol.) Later, you might also find a way to pick up data off the ethernet based network that Furuno uses, or the CAN-based network in NMEA-2000.

If you define a standard data interchange format, you can have arbitrarily many different programs to collect the raw data. It could be possible to collect the data with a PDA, even.

There are a bunch of problems. The answers mostly seem pretty obvious, except for how to deal with malicious data. I can't think of a way to prevent somebody sending you bad data specifically to cause you problems.
__________________
Mark S.
coot is offline   Reply With Quote
Old 06-06-2006, 22:57   #10
Registered User

Join Date: Feb 2006
Location: Portland, Maine
Posts: 24
Send a message via AIM to MaineCub Send a message via Yahoo to MaineCub
Coot;

WOW! Great and thoughtful response. You are right, malicious data is a significant obstacle. Some things I've thought of.

1. check-digits. An algorithm used by the application to make sure all of the data in a record is in sync. Used alot in the banking industry (ie: for checking account numbers)

2. Outliers: I studied this in grad school (sociology), responses that were so many standard deviations from the "average" were discarded.

These are just ideas. I by no means have the answers to all of these. Like I said, my eyes are bigger than my stomach!
MaineCub is offline   Reply With Quote
Old 07-06-2006, 11:21   #11
Registered User
 
DeepFrz's Avatar

Cruisers Forum Supporter

Join Date: Mar 2006
Location: Winnipeg
Boat: None at this time
Posts: 8,462
Hi all;
Malicious damage to a data base is something that always has to be guarded against. I know that several of the open source projects that I worked on had "mentors" that we could send our work to untill we became trusted to post to the CVS system. A (perhaps) bigger problem that I have seen in open source is someone hi-jacking the data and going commercial while leaving the source code a mess. It is still available but in essentially a non-working state. Flight Log is such a case. ArgoUML was not quite as bad, however it was basically hi-jacked with very little working code returned to the source while the commercial enterprise scanned all new code as it was submitted.

That being said, open source has produced some remarkably good software. Maybe even better than "most" commercially produced stuf. The main thing is protecting the source code with sufficient security and back ups and closely monitoring the code as it is submitted. CVS "branches" are a big help in this as is using Linux for the server.

Dave, I used the term VCS as in Version Control System, meaning a non-specific type of vcs rather than the specific CVS.

Phil.
DeepFrz is offline   Reply With Quote
Old 07-06-2006, 12:45   #12
Registered User

Join Date: Feb 2006
Location: Portland, Maine
Posts: 24
Send a message via AIM to MaineCub Send a message via Yahoo to MaineCub
This is all good.. But rather than get muddled down with all of the complexities of this type of project...........

Most open source projects seem to start with one basic idea, then growing/expanding when needed.

I think the biggest thing is what is said before, clearly defining the problems and requirements of this project. In addition, designing the framework such that it can be easily expanded.

Im bored at work.. Im skipping out of here early and going home to load up Fedora / Subversion / MySQL / JBoss onto my server.

I'll let you know how this works out.

Best,

--Dave
MaineCub is offline   Reply With Quote
Old 11-06-2006, 22:13   #13
Registered User
 
coot's Avatar

Join Date: Feb 2006
Posts: 367
Images: 2
Quote:
Originally Posted by DeepFrz
Hi all;
Malicious damage to a data base is something that always has to be guarded against. I know that several of the open source projects that I worked on had "mentors" that we could send our work to untill we became trusted to post to the CVS system.
I'm not that concerned with code integrity (well I am, but that problem has been solved for decades).

Presumably, this
Quote:
The recorder could then on a timely basis report it's data back to a central location. Subscribers could download the data into their navigation machines etc and use.
means that you would accept data from multiple sources, with the intent that users would use data that they did not generate themselves.

Since your project is open source, any random loser can create random data that looks exactly like real data, but has no basis in reality. Your database could identify the contributor for each data point, but depending how much verification you require, that may not be very useful. For example, after you run aground on rocks that are 10 feet shallower than the database indicated, knowing the bad data came from 31337@hotmail.com doesn't help much.

If you make a more formal relationship with the data contributors, you could verify real identities. For example, if you meet each contributor in person and make copies of government issued IDs, you have quite a bit more confidence that you know where the data comes from. In that case, the risk of personal liability may prevent someone from uploading intentionally misleading data. (It may also prevent someone from uploading perfectly valid data...)

Having established an identity for each contributor, you could use public key or shared secret authentication to prove that the data came from the person it is supposed to have come from. The central data repository would want to track the source for every point in the database. This would be useful to purge data from sources later determined to be untrustworthy.

It also would allow corrections to the data based on who collected it. For example, the US National Weather Service accepts weather data from ships at sea through their VOS program. Over time, they tend to identify bias of individual observers. For example, "their guy isn't good at estimating wave height and always comes in a little high" or "their barometer calibration isn't quite right". After they identify those biases, they apply corrections to reports from that ship.

Comparing data from multiple sources can help identify bad (intentional or not) data. The problem is you need multiple overlapping sources, which may be difficult for this kind of project. I have not noticed boaters to have much overlap with users of open source systems.

One thing I think would be very interesting would be a comparison of data from the same location collected several times by the same source. That is, how closely can I match my own data?

One way to bring less computer savvy users into your system is to provide easy-to-use binaries for the popular computers you expect to have on a boat. For example, just about all the recreational navigation software I've seen runs on MS Windows. If you have a data collector that runs on Windows without interfering with the nav software, you have a lot more potential users. If your data download can be easily converted into ENC charts or bit maps for overlays, it would be a real win. This gives you a way to get data from people who would not otherwise be able to participate at all.
__________________
Mark S.
coot is offline   Reply With Quote
Old 12-06-2006, 08:13   #14
Moderator Emeritus
 
Pblais's Avatar

Join Date: Mar 2003
Location: Hayes, VA
Boat: Gozzard 36
Posts: 8,700
Images: 15
Send a message via Skype™ to Pblais
Quote:
It also would allow corrections to the data based on who collected it.
Actually all the data is 'uncorrectable'. An more important none of the data collected is collected in anything more than a bit less than random pattern. For example should I have collected data of every place I have been in the last 3 years (I have most of it) what would you have? Most of the data would be duplicated from places I have retraced all too many times. The rest is of places charted with known information.

The process of making collected data into something that could be useful for someone else is the critical part of the solution. You can't average the data collected as the factors required for correction are all unknown. Monkeys at typewriters becomes a closer analogy. The monkeys have an advantage in that they have a finite number of keys.

The collection of data is more a structured process than a mass random sample. Consider the Pacific Island charts created during the early 19th century by the US Exploring Expedition. Many of these charts were the sole data used in WW II and were quite accurate and still are. They were made accurate by the process used to make the charts and not by large samples of data collected. It was a well thought out collection of a very very small number of points and correlated through mathematical baselines. The goal was a complete chart not just a few points. A few of the important points means more than the number of points. You proposed to chart the deep trenches but never locate the true "end of the shoal".

The process of navigational chart creation is more than a collection of uncorrelated and uncorrected GPS points with poorly collected depth soundings. At most you will have a collection of popularly traveled "channels" that falls very short of a real chart and lack most of the real information you need to call it a chart - even it you accomplish what you state is the goal. I would even conclude you could do it too.

Currently you can find sites that offer "way point" collections. At least were I to upload a route of 25 way points that could navigate you up a shallow channel 9 nm long it may be of use to someone were they to know that the information provided was indeed used by a similar vessel and was for that specific purpose. A collection of every place I have ever been would include that data but it's lost in a sea of meandering routes.

You could examine the data but you could not see all the places I made a "mistake" even if we could assume my data was 100 % accurate. I often go the long way around where I did not know I could have taken a short cut. You of course won't ever know that from my taced tracks. I often construct way point routes using the data I collect but it is also corrected to include a whole lot of other factors beside a retrace of the places I went yet the GPS data is all 100% raw.

Navigation is more than just the collection of points but more a creation of a route through a complete data rich chart where I can accommodate leeway for the shoals and places I know will be subject to depths that may have changed or navigation aids I choose to verify my position against or the day marks of the channel in case it is dark or foggy when I attempt the route. In short, I don't not only rely on the routed way points alone.

I require more information than a set of way points alone. You offer even less than that. Your goal of collecting data points is doomed not because you can't accomplish the task but because the task you intend falls far short of the goal. You can't win because you are not really in the game.

Collecting NMEA data to disk is a trivial exercise. You can purchase software for almost nothing. The NMEA sentance structure is well documented and can be collected with a serial port running at pretty slow speeds. Open source code is not required. Making the collected data into a chart requires far more than this and in reality if you had everything else you wouldn't need the data you are collecting. It is as if you are building a boat and have come up with a scheme for getting the bottom paint.
__________________
Paul Blais
s/v Bright Eyes Gozzard 36
37 15.7 N 76 28.9 W
Pblais is offline   Reply With Quote
Old 12-06-2006, 11:32   #15
Registered User
 
DeepFrz's Avatar

Cruisers Forum Supporter

Join Date: Mar 2006
Location: Winnipeg
Boat: None at this time
Posts: 8,462
I don't know how people got the idea that Dave has been talking about chart making. This is about writing Open Source software to be used for navigational purposes, not creating charts. The data in this instance is "code", not chart data points. We were talking about CVS/Subversion for a software code depository and setting up an IDE and data base system to get started. I guess it's nice to know how charts were plotted in the old days if one were interested in that sort of thing.

Deep.
DeepFrz is offline   Reply With Quote
Reply

Tags
navigation

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Free navigation software JanPeter Navigation 33 17-04-2009 17:05
Celestial Navigation SkiprJohn Navigation 45 29-12-2008 22:15
Open for Suggestions... Greg General Sailing Forum 30 24-11-2004 22:06
Source for paper charts sjs General Sailing Forum 7 18-03-2004 01:44
Open Cabin -- August BVI Charter Kathie_WE Atlantic & the Caribbean 0 15-06-2003 19:03

Advertise Here


All times are GMT -7. The time now is 19:14.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2024, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.