Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Search this Thread Rate Thread Display Modes
Old 19-02-2018, 19:46   #1
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 5,145
MBTiles for OpenCPN

Just when you thought it was safe to take a little break from OCPN.....

It is time to introduce MBTiles support in OpenCPN.
We are planning some level of support for MBTiles on OpenCPN Version 5.x, due out later this year. Full details are not yet determined.

To get started, we have prototyped native built-in support for MBTiles in the development tree. Source builders may find the code in github master branch now. Mac is untested so far. Windows users wanting to experiment may find a precompiled (but unsupported) Alpha version here:

https://www.dropbox.com/s/ljdtl742ux...setup.exe?dl=0

Notes:
1. OpenGL is required, at least for now. May be required for reasonable performance in production. We shall see.
2. MBTiles tilesets may be added to OCPN chart directories just like any other chart, in this case using the "mbtiles" file name extension.

So, where does one find MBTiles tilesets?
There are some precompiled tilesets floating around the internet. NOAA, for example, provides a download service for pre-built tilesets covering their range of RNCs. These are good tilesets, although quite large with many, many zoom levels included. There are others, of unknown provenance. Google around to find them, and decide for yourself if they are suitable for navigation.

On the other hand, many users will want to use some external tool to build tilesets for their navigational area of interest. Photo overlays come to mind here. I have experimented with SASPlanet, and find it quite useful for capturing tilesets of many popular and some more obscure cartographies. This Windows app downloads charts from many sources, and creates quilted tilesets directly from one user interface. Nice.

Our goal here:
Experimentation. What works, what can these things be used for, and how will this be productized for OCPN5.

As we move forward, we will talk more about the advantages and disadvantages of MBTiles. Those already having some experience with this chart format are encouraged to tell us what you know.

As always, I appreciate your continued support and feedback.
Dave
__________________

bdbcat is offline   Reply With Quote
Old 19-02-2018, 21:26   #2
Registered User

Join Date: Feb 2010
Location: On the go. Not in Prague.
Posts: 4,497
Re: MBTiles for OpenCPN

Dave...
Very nice!

To save myself some pain, I've used the bundled sqlite3 also on macOS - With the system wide one the linker complained about sqlite3_load_extension (Different error than the one seen in the Travis build)

And this must be my bad karma - Grabbed the first dataset found on the net - https://openmaptiles.com/downloads/d...es/#0.23/0/-26 - metadata say format is JPG, but the actual tiles are PNG...

Fixing that, nirvana (Until a GL nullptr crash a while later).

This will once again be fun )))

Pavel
__________________

nohal is offline   Reply With Quote
Old 19-02-2018, 21:51   #3
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 5,145
Re: MBTiles for OpenCPN

nohal...

I see no reason not to use the embedded sqlite3 on Mac. But you know what the debian guys would say for linux....


GL Crash? Any further info?

Dave
bdbcat is offline   Reply With Quote
Old 19-02-2018, 22:22   #4
Registered User

Join Date: Nov 2007
Location: Probably in an anchorage or a boatyard..
Boat: Ebbtide 33' steel cutter
Posts: 3,931
Re: MBTiles for OpenCPN

Looks really handy! Thanks.

WHat seetings are you using in sasplanet? Can't seem to get it to work.



Works ok with this file -
https://gitlab.com/IvanSanchez/Leafl...raster.mbtiles
conachair is offline   Reply With Quote
Old 19-02-2018, 23:43   #5
Registered User

Join Date: Jun 2015
Posts: 270
Re: MBTiles for OpenCPN

Quote:
Originally Posted by bdbcat View Post
Just when you thought it was safe to take a little break from OCPN.....

It is time to introduce MBTiles support in OpenCPN.
We are planning some level of support for MBTiles on OpenCPN Version 5.x, due out later this year. Full details are not yet determined.

To get started, we have prototyped native built-in support for MBTiles in the development tree. Source builders may find the code in github master branch now. Mac is untested so far. Windows users wanting to experiment may find a precompiled (but unsupported) Alpha version here:

https://www.dropbox.com/s/ljdtl742ux...setup.exe?dl=0

Notes:
1. OpenGL is required, at least for now. May be required for reasonable performance in production. We shall see.
2. MBTiles tilesets may be added to OCPN chart directories just like any other chart, in this case using the "mbtiles" file name extension.


On the other hand, many users will want to use some external tool to build tilesets for their navigational area of interest. Photo overlays come to mind here. I have experimented with SASPlanet, and find it quite useful for capturing tilesets of many popular and some more obscure cartographies. This Windows app downloads charts from many sources, and creates quilted tilesets directly from one user interface. Nice.

Our goal here:
Experimentation. What works, what can these things be used for, and how will this be productized for OCPN5.

As we move forward, we will talk more about the advantages and disadvantages of MBTiles. Those already having some experience with this chart format are encouraged to tell us what you know.

As always, I appreciate your continued support and feedback.
Dave
My main issue with mbtiles is there's no overzoom you can get/produce 1/3 000 tiles but if the source is 1/40 000 they will be wrong and you won't know it. Anyway it's already the case as you can find/produce these same bogus charts in kap format.

Couple of things:
- It's crazy but at least on linux debian libcurl is already pulling sqlite, using an internal version is likely asking for big troubles, same for plugins using it.

- CMake voodoo doesn't seem to work 100%; it doesn't stop on missing libsqlite3-dev but later make can't compile SQLiteCpp.

- Is it possible to use a git subtree for SQLiteCpp rather than a code dump?

Unlike submodules subtrees make no difference for other users/devs, with --squash options it doesn't add much to project history but it makes a big one when trying to updating it later.
There wasn't subtree command then but for example there are bugfixes in gdal upstream which are very hard to pull back.


Thread drift but as 5.0 is going to break all plugins with wx what about adding a second abstraction layer? Plugins using it will be only able to run against a specific version, O already has API for requesting its version, it would remove code duplicated in internal plugins and some plugins already only work with a specific O version anyway. May force more plugins to use a proper namespace as in BR24radar_pi, not a bad thing.
did-g is offline   Reply With Quote
Old 19-02-2018, 23:46   #6
Registered User

Join Date: Sep 2012
Location: Baikal
Posts: 319
Re: MBTiles for OpenCPN

Great news!
For a long time asked for this!

Works quickly.
There are problems with displaying on scales (the map view disappears) if several files are open.

Why is it not taken into account the zoom more than 18 (sasplanet)

for tests - SatImg WorldMap zoom 5-9 (sasPlanet)
https://yadi.sk/d/6lfjJOaA3SbFuR
Baikal is offline   Reply With Quote
Old 20-02-2018, 01:35   #7
Registered User

Join Date: Mar 2017
Location: Perros-Guirec, France
Boat: Dufour 31
Posts: 239
Re: MBTiles for OpenCPN

FYI, compiling the lastest git master on recent linux is likely to fail because of an incompatibility between the version of libsqlite3-dev (>=3.19) and the usage of -DSQLITE_USE_LEGACY_STRUCT in OpenCPN.

While waiting for a proper fix (detecting the system libsqlite3 version and adjusting the build flags accordingly), you can edit CMakeLists.txt and remove the
Code:
add_definitions(-DSQLITE_USE_LEGACY_STRUCT)
line
stelian is offline   Reply With Quote
Old 20-02-2018, 02:36   #8
Registered User

Join Date: Aug 2016
Posts: 33
Re: MBTiles for OpenCPN

Hi Dave,

thx for this feature in OpenCPN.

I have tried latest Build from AppVoyer and I haven't luck

I have tried both schemes ( xyz and tms) of mbtiles. Neither nor seems doesn't work

Quote:
11:16:41: Loading chart data for ..\..\Charts\mbtiles\world.mbtiles
11:16:41: Adding chart file: ..\..\Charts\mbtiles\world.mbtiles
11:16:45: Finished chart database Update
Quote:
11:18:03: Loading chart data for ..\..\Charts\mbtiles\worldtms.mbtiles
11:18:03: Adding chart file: ..\..\Charts\mbtiles\worldtms.mbtiles
11:18:03: Finished chart database Update
Both files provides zooms 3,4,6,7

But I see only GSHHS Charts

If you want, I can provide both Files

Regards
BlackSea
BlackSea is offline   Reply With Quote
Old 20-02-2018, 03:41   #9
Senior Cruiser
 
sailorF54's Avatar

Cruisers Forum Supporter

Join Date: Dec 2009
Location: Perros-Guirec, France
Boat: Jeanneau Sunshine 36
Posts: 923
Re: MBTiles for OpenCPN

Quote:
Originally Posted by stelian View Post
FYI, compiling the lastest git master on recent linux is likely to fail because of an incompatibility between the version of libsqlite3-dev (>=3.19) and the usage of -DSQLITE_USE_LEGACY_STRUCT in OpenCPN.

While waiting for a proper fix (detecting the system libsqlite3 version and adjusting the build flags accordingly), you can edit CMakeLists.txt and remove the
Code:
add_definitions(-DSQLITE_USE_LEGACY_STRUCT)
line
No luck with that fix (commenting out or deleting the line) , still
Code:
/home/pytheas/OpenCPN/src/SQLiteCpp/src/Backup.cpp:16:21: fatal error: sqlite3.h: Aucun fichier ou dossier de ce type
compilation terminated.
CMakeFiles/SQLiteCpp.dir/build.make:65 :  la recette pour la cible  « CMakeFiles/SQLiteCpp.dir/src/SQLiteCpp/src/Backup.cpp.o » a échouée
make[2]: *** [CMakeFiles/SQLiteCpp.dir/src/SQLiteCpp/src/Backup.cpp.o] Erreur 1
make[2] : on quitte le répertoire « /home/pytheas/OpenCPN/build »
CMakeFiles/Makefile2:203 : la recette pour la cible « CMakeFiles/SQLiteCpp.dir/all » a échouée
make[1]: *** [CMakeFiles/SQLiteCpp.dir/all] Erreur 2
make[1] : on quitte le répertoire « /home/pytheas/OpenCPN/build »
Makefile:152 : la recette pour la cible « all » a échouée
make: *** [all] Erreur 2
sailorF54 is offline   Reply With Quote
Old 20-02-2018, 03:47   #10
Registered User

Join Date: Mar 2017
Location: Perros-Guirec, France
Boat: Dufour 31
Posts: 239
Re: MBTiles for OpenCPN

Quote:
Originally Posted by sailorF54 View Post
No luck with that fix (commenting out ou deleting the line) , still
[CODE]/home/pytheas/OpenCPN/src/SQLiteCpp/src/Backup.cpp:16:21: fatal error: sqlite3.h: Aucun fichier ou dossier de ce type
compilation terminated.
Off course, you also need libsqlite3 development libraries installed:

Code:
apt-get install libsqlite3-dev
(on Debian/Ubuntu/Rasbian etc, adapt for other distributions)
stelian is offline   Reply With Quote
Old 20-02-2018, 03:56   #11
Senior Cruiser
 
sailorF54's Avatar

Cruisers Forum Supporter

Join Date: Dec 2009
Location: Perros-Guirec, France
Boat: Jeanneau Sunshine 36
Posts: 923
Re: MBTiles for OpenCPN

A single Mbiles chart (zoom level 14 to 17) file displays fine with W10.
Trying to split the same zone in 4 different charts (zoom 14, 15...) with/without quilting creates havoc: not display, boat sent to 0,0 etc...
(The idea being to be able to zoom in /out on the various levels)
sailorF54 is offline   Reply With Quote
Old 20-02-2018, 04:07   #12
Senior Cruiser
 
sailorF54's Avatar

Cruisers Forum Supporter

Join Date: Dec 2009
Location: Perros-Guirec, France
Boat: Jeanneau Sunshine 36
Posts: 923
Re: MBTiles for OpenCPN

Quote:
Originally Posted by stelian View Post
Off course, you also need libsqlite3 development libraries installed:

Code:
apt-get install libsqlite3-dev
(on Debian/Ubuntu/Rasbian etc, adapt for other distributions)


but
Code:
In file included from /home/pytheas/OpenCPN/src/SQLiteCpp/src/Backup.cpp:16:0:
/usr/include/sqlite3.h:3455:20: error: conflicting declaration ‘typedef struct Mem sqlite3_value’
 typedef struct Mem sqlite3_value;
                    ^
In file included from /home/pytheas/OpenCPN/src/SQLiteCpp/include/SQLiteCpp/Backup.h:14:0,
                 from /home/pytheas/OpenCPN/src/SQLiteCpp/src/Backup.cpp:12:
/home/pytheas/OpenCPN/src/SQLiteCpp/include/SQLiteCpp/Database.h:23:30: note: previous declaration as ‘typedef struct sqlite3_value sqlite3_value’
 typedef struct sqlite3_value sqlite3_value;
                              ^
CMakeFiles/SQLiteCpp.dir/build.make:65 : la recette pour la cible « CMakeFiles/SQLiteCpp.dir/src/SQLiteCpp/src/Backup.cpp.o » a échouée
sailorF54 is offline   Reply With Quote
Old 20-02-2018, 04:10   #13
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,339
Re: MBTiles for OpenCPN

Quote:
Originally Posted by stelian View Post
FYI, compiling the lastest git master on recent linux is likely to fail because of an incompatibility between the version of libsqlite3-dev (>=3.19) and the usage of -DSQLITE_USE_LEGACY_STRUCT in OpenCPN.

While waiting for a proper fix (detecting the system libsqlite3 version and adjusting the build flags accordingly), you can edit CMakeLists.txt and remove the
Code:
add_definitions(-DSQLITE_USE_LEGACY_STRUCT)
line
On Ubuntu 16.04 it's alright to compile after installing "libsqlite3-dev".
The version is only 3.11.0, so it's old enough to work.

Thomas
cagney is offline   Reply With Quote
Old 20-02-2018, 04:24   #14
Registered User

Join Date: Mar 2017
Location: Perros-Guirec, France
Boat: Dufour 31
Posts: 239
Re: MBTiles for OpenCPN

Quote:
Originally Posted by sailorF54 View Post
previous declaration as ‘typedef struct sqlite3_value sqlite3_value
François, read my reply #7 above...
stelian is offline   Reply With Quote
Old 20-02-2018, 04:40   #15
Senior Cruiser
 
sailorF54's Avatar

Cruisers Forum Supporter

Join Date: Dec 2009
Location: Perros-Guirec, France
Boat: Jeanneau Sunshine 36
Posts: 923
Re: MBTiles for OpenCPN

Quote:
Originally Posted by stelian View Post
François, read my reply #7 above...
But I have done it...
(i.e. deleting the line)
__________________

sailorF54 is offline   Reply With Quote
Reply

Tags
enc, opencpn

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
mbtiles in opencpn boat_alexandra OpenCPN 15 02-10-2017 01:36
MBTILES for opencpn ploubaz22 OpenCPN 4 25-12-2016 06:50
Mbtiles edelvoilier OpenCPN 3 23-05-2016 13:02
Virtual OpenCPN - 'OpenCPN on a Stick' r.fairman OpenCPN 23 16-10-2011 19:51



Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 15:58.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2018, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.