Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Rate Thread Display Modes
Old 16-02-2010, 18:18   #31
Registered User
 
HamishB's Avatar

Join Date: Jan 2010
Location: New Zealand
Posts: 286
Quote:
Originally Posted by hackoon View Post
SpatialLite seems to be to good to be true. What arguments are left against sqlite integration?
It's neat tech alright, but is there any actual demand for its features in the code? Is the current implementation lacking in some way which would be solved by adding it? How would it make things better?

i.e., we could add support for it, but what's the point? What will it allow you to do which you can't do now?

are you suggesting to have it hold the chart file database?
or have it hold a new waypoint management system?


Hamish
(if it aint broke..)
__________________

__________________
HamishB is offline   Reply With Quote
Old 18-02-2010, 04:58   #32
Registered User
 
hackoon's Avatar

Join Date: Jan 2010
Posts: 39
It is not a feature by itself but an important basis for further development (please see the whole discussion in this thread and "googleearth for sailors")
__________________

__________________
hackoon is offline   Reply With Quote
Old 18-02-2010, 06:17   #33
cruiser

Join Date: Oct 2007
Posts: 751
Developers often get confused in the process of adding features between choosing the things they CAN add versus the things they SHOULD add. This thread has been interesting and a good discussion. There's been a fair amount of passive aggressiveness that's unhealthy for the discussion but other than that, some excellent points have been brought up.

The storage requirements for every part of chartplotter software grow with use and features. Really use this type of software for just a moderate amount of time and the data demands skyrocket. I'll bet I have 350 routes that I created averaging some 20 - 100 waypoints each. Tracks add exponentially to that. POI's add. Even the charts themselves become data points in need of management. Want to search for place names? That adds gigabytes of structured data. Add pilot guides? More data, more management. Weather? Buoy reports? Tide and current? Data, data, data.

Writing the management code for each one of those elements is a major job that most probably won't be designed right the first time around. Either data needs or performance results will show up later and you'll be off re-writing code again, delaying more important features, and binding the projects into a quagmire.

Using an SQL database (SQLite, SpatialLite, whatever) is such an obvious architecture decision. It isn't about profiling the code now. It's about allowing growth now. There's no tool that can give you an analysis of that - just your own ability to see around. Why has every other major charting software gone to SQL database support for their data?

Someone previously had a great posting. It was something to the effect of, if it'll be hard to integrate now, imagine how hard it will be when there are more features and more code in the project.

Building SQL data support into and throughout the code goes far beyond what CAN or what SHOULD be done. It's obvious to me that it HAS TO be done.
__________________
ActiveCaptain is offline   Reply With Quote
Old 18-02-2010, 08:47   #34
Registered User

Join Date: Nov 2009
Location: France
Posts: 63
Hi Jeffrey,

I'm glad to see you are there to explain theses important things to confused programmers.
You could also explain to confused users that programmers may code only precise features or loose much time to code and recode again.

SQL is not a response for anything and there are many ways it could be implemented.
Among other things, it depends on data volume, data types and what you want to do with data.
And there are data that don't have to go inside a database.
It's the reason why SQL databases exist but are not the only method.
A SQL database also costs time and /or money (DB administration, utilities for maintenance, disk room, etc).
Some developpers are aware of this kind of things. So we should let them decide about 'how'.

So :
What features would you like exactly to be implemented into OCPN ?

Jean-Pascal
__________________
Totobeloeil is offline   Reply With Quote
Old 18-02-2010, 09:37   #35
cruiser

Join Date: Oct 2007
Posts: 751
Quote:
Originally Posted by Totobeloeil View Post
What features would you like exactly to be implemented into OCPN ?
That's outside the scope of this thread and I wouldn't want to de-rail it.

Architecture isn't about features. It's about design. And future features. And making the development of future features easier.
__________________
ActiveCaptain is offline   Reply With Quote
Old 18-02-2010, 11:40   #36
Registered User
 
Psyches's Avatar

Join Date: Apr 2008
Location: SF Bay Area
Boat: Tartan 30 - Bluegrass
Posts: 187
Send a message via Yahoo to Psyches Send a message via Skype™ to Psyches
Quote:
Originally Posted by hackoon View Post
Who is out there willing to help w'mith a sqlite integration/rewrite of opencpn? I can do a part of it but next few months I sailing into some wild rivers (iridium conection only!).
I can almost hear the thundering crowd of developers running to respond, ready to donate months of their time to transform this working product to a possibly even better one I again stand by what I said, but I agree with Hamish..."if it ain't broke..."

Quote:
Originally Posted by ActiveCaptain View Post
Developers often get confused in the process of adding features between choosing the things they CAN add versus the things they SHOULD add. This thread has been interesting and a good discussion. There's been a fair amount of passive aggressiveness that's unhealthy for the discussion but other than that, some excellent points have been brought up.
Possibly directed to me, and that's ok...I can be that way. But I'd like to think I have a few insights and some solid pragmatism, and few decades of experience to back it up. Nevertheless, I try to be honest, and have and continue to invite those with time and interest to explore this area. And I've been wrong before, I have no problem admitting it.

Jeff you have some good points further down in that post - I've not said we should never use a database; SQL is a choice, but as Jean-Pascal said there are other possibilities, and there are issues that perhaps you don't see from your position? Just a possibility?

Quote:
Originally Posted by ActiveCaptain View Post
Architecture isn't about features. It's about design. And future features. And making the development of future features easier.
Absolutely. But armchair architecture without some reasonable understanding of the underlying code base is not only useless, it's dangerous. Good architectural decision-making factors in how the code base really is, what's difficult to do, what's important to do, and what the capabilities are of the team. The point about "hard now, harder later" is a good one, if not completely correct - we need to keep our eyes open, for sure.

If Dave came in and said "ok, let's toss the basic storage changes from the last half-year and rewrite" I'd be 100% for it. Make that 150%. But that's because he is the "owner", the lead, has a solid understanding of the code, and a proven commitment to the completion of projects. Anyone else saying it, including myself, frankly, is just talk...and talk is cheap.

And the fact is, there are things we really can do to vastly improve OpenCPN without the big re-write...for now.

Mark
__________________
Psyches is offline   Reply With Quote
Old 18-02-2010, 18:54   #37
Registered User
 
scotte's Avatar

Join Date: Apr 2007
Location: SF Bay Area, CA, USA
Boat: Privilege 39
Posts: 664
SQL is a tool, but sometimes it's like bringing a bazooka to a knife fight. As developers, we tend to implement what we are familiar+comfortable with, but there's also a big difference between "a problem in search of a solution" and "a solution in search of a problem". I'm afraid we are loaded towards the latter in this case.

I've been using SQL professionally for about 20 years, but it's seldom my first choice. These days, with good design you can do more, and faster, with much less. There are some fast and simple key/value stores that are a much better choice than SQL - DB naysayers tend to disagree, but that's expected from folks who think of everything in terms of rows and columns, instead of keys and documents. And then sometimes, quite often, resorting to a flat or structured file is just the best way to go.

The point is, first you have to work out the requirements, and what it is, exactly, you are trying to build. Only then can you choose the appropriate technology to implement it. Look at how mechanical engineers do their thing, you don't see them creating random bits and then trying to bolt+weld them together... We software engineers need to do what they do!
scotte is offline   Reply With Quote
Old 20-02-2010, 04:10   #38
Registered User
 
hackoon's Avatar

Join Date: Jan 2010
Posts: 39
What knife fight please? This is not tetris and there is no doubt that the amounts of data need some kind of data-management. I'm happy to check out alternatives. We don't need a tank but bazooka is maybe the right choice.
The random bits is exactly what is happening to this point!
__________________
hackoon is offline   Reply With Quote
Old 12-03-2010, 19:24   #39
Registered User

Join Date: Jan 2010
Posts: 32
Target hardware, and a storm in a teacup?

Quote:
Originally Posted by Netsurfer View Post
Another point to keep in mind for further developement is that OCPN might not only be used on "high-end" Notebooks (just think about Netbooks).
In my opinion the option to use OCPN on small Netbooks and "low-end" NB should have the/ one of the highest prioritie(s).
I've not seen problems with OpenCPN performance and high usage levels of waypoints, routes and tracks, but if the data is stored using XML then I would anticipate serious problems on anything but very high specification machines. XML is not meant for anything other than serialised transfer of datastructures between not-necessarily-compatible architecture systems. Any application that uses extensive XML schemas for storing moderate data volumes and performing even vaguely significant processing will likely be very badly hindered indeed, performance-wise.

Realistically, if you look at the hardware that very many people are going to be running OpenCPN on in the next 2 to 3 years, this is going to be a problem.

There are hordes of Net-tablets coming. They are going to be cheap and ubiquitous, and will run Linux of various flavours (as well as Android, although that presents different issues). They are the hardware that a huge number of OpenCPN users will want to be using.

These devices will be running on relatively modestly powered ARM processors, and storage will be onboard flash or SDHC card.

Storing large volumes of constantly accessed / updated data in XML is not the way to go with target hardware like that.

I appreciate the arguments about re-work being hard, but I don't believe anyone is talking about replacing the system config files, or moving charts into BLOBs within SQLite. The only objective at present surely is to get the problem datasets out of the XML config file and into a half dozen or so SQLite tables. It's not rocket science, and could barely be classed as extensive or risky re-engineering work?

Regards,

Mike
__________________
bluearcus is offline   Reply With Quote
Old 12-03-2010, 23:59   #40
Registered User

Join Date: Nov 2009
Location: France
Posts: 63
Quote:
Originally Posted by bluearcus View Post
It's not rocket science, and could barely be classed as extensive or risky re-engineering work?
So, what ? Just do it !
__________________
Totobeloeil is offline   Reply With Quote
Old 13-03-2010, 02:20   #41
Senior Cruiser
 
idpnd's Avatar

Cruisers Forum Supporter

Join Date: Sep 2007
Location: Almerķa, ES
Boat: Chiquita 46 - Libertalia
Posts: 1,551
Hello Bluearcus,

there's nothing left to be discussed here really, one of the lead devs has stated (above) its fine for anyone particularly keen on the sql implementation to do one.
__________________

__________________
sv Libertalia
idpnd is offline   Reply With Quote
Reply

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off




Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 02:44.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.