MBTiles for OpenCPN

[bdbcat]

Just when you thought it was safe to take a little break from OCPN.....

It is time to introduce MBTiles support in OpenCPN.
We are planning some level of support for MBTiles on OpenCPN Version 5.x, due out later this year. Full details are not yet determined.

To get started, we have prototyped native built-in support for MBTiles in the development tree. Source builders may find the code in github master branch now. Mac is untested so far. Windows users wanting to experiment may find a precompiled (but unsupported) Alpha version here:

https://www.dropbox.com/s/ljdtl742ux...setup.exe?dl=0

Notes:
1. OpenGL is required, at least for now. May be required for reasonable performance in production. We shall see.
2. MBTiles tilesets may be added to OCPN chart directories just like any other chart, in this case using the "mbtiles" file name extension.

So, where does one find MBTiles tilesets?
There are some precompiled tilesets floating around the internet. NOAA, for example, provides a download service for pre-built tilesets covering their range of RNCs. These are good tilesets, although quite large with many, many zoom levels included. There are others, of unknown provenance. Google around to find them, and decide for yourself if they are suitable for navigation.

On the other hand, many users will want to use some external tool to build tilesets for their navigational area of interest. Photo overlays come to mind here. I have experimented with SASPlanet, and find it quite useful for capturing tilesets of many popular and some more obscure cartographies. This Windows app downloads charts from many sources, and creates quilted tilesets directly from one user interface. Nice.

Our goal here:
Experimentation. What works, what can these things be used for, and how will this be productized for OCPN5.

As we move forward, we will talk more about the advantages and disadvantages of MBTiles. Those already having some experience with this chart format are encouraged to tell us what you know.

As always, I appreciate your continued support and feedback.
Dave

[nohal]

Dave...
Very nice!

To save myself some pain, I've used the bundled sqlite3 also on macOS - With the system wide one the linker complained about sqlite3_load_extension (Different error than the one seen in the Travis build)

And this must be my bad karma - Grabbed the first dataset found on the net - https://openmaptiles.com/downloads/d...es/#0.23/0/-26 - metadata say format is JPG, but the actual tiles are PNG...

Fixing that, nirvana (Until a GL nullptr crash a while later).

This will once again be fun )))

Pavel

[bdbcat]

nohal...

I see no reason not to use the embedded sqlite3 on Mac. But you know what the debian guys would say for linux....


GL Crash? Any further info?

Dave

[conachair]

Looks really handy! Thanks.

WHat seetings are you using in sasplanet? Can't seem to get it to work.



Works ok with this file -
https://gitlab.com/IvanSanchez/Leafl...raster.mbtiles

[did-g]

Originally Posted by bdbcat

Just when you thought it was safe to take a little break from OCPN.....

It is time to introduce MBTiles support in OpenCPN.
We are planning some level of support for MBTiles on OpenCPN Version 5.x, due out later this year. Full details are not yet determined.

To get started, we have prototyped native built-in support for MBTiles in the development tree. Source builders may find the code in github master branch now. Mac is untested so far. Windows users wanting to experiment may find a precompiled (but unsupported) Alpha version here:

https://www.dropbox.com/s/ljdtl742ux...setup.exe?dl=0

Notes:
1. OpenGL is required, at least for now. May be required for reasonable performance in production. We shall see.
2. MBTiles tilesets may be added to OCPN chart directories just like any other chart, in this case using the "mbtiles" file name extension.


On the other hand, many users will want to use some external tool to build tilesets for their navigational area of interest. Photo overlays come to mind here. I have experimented with SASPlanet, and find it quite useful for capturing tilesets of many popular and some more obscure cartographies. This Windows app downloads charts from many sources, and creates quilted tilesets directly from one user interface. Nice.

Our goal here:
Experimentation. What works, what can these things be used for, and how will this be productized for OCPN5.

As we move forward, we will talk more about the advantages and disadvantages of MBTiles. Those already having some experience with this chart format are encouraged to tell us what you know.

As always, I appreciate your continued support and feedback.
Dave

My main issue with mbtiles is there's no overzoom you can get/produce 1/3 000 tiles but if the source is 1/40 000 they will be wrong and you won't know it. Anyway it's already the case as you can find/produce these same bogus charts in kap format.

Couple of things:
- It's crazy but at least on linux debian libcurl is already pulling sqlite, using an internal version is likely asking for big troubles, same for plugins using it.

- CMake voodoo doesn't seem to work 100%; it doesn't stop on missing libsqlite3-dev but later make can't compile SQLiteCpp.

- Is it possible to use a git subtree for SQLiteCpp rather than a code dump?

Unlike submodules subtrees make no difference for other users/devs, with --squash options it doesn't add much to project history but it makes a big one when trying to updating it later.
There wasn't subtree command then but for example there are bugfixes in gdal upstream which are very hard to pull back.


Thread drift but as 5.0 is going to break all plugins with wx what about adding a second abstraction layer? Plugins using it will be only able to run against a specific version, O already has API for requesting its version, it would remove code duplicated in internal plugins and some plugins already only work with a specific O version anyway. May force more plugins to use a proper namespace as in BR24radar_pi, not a bad thing.

[Baikal]

Great news!
For a long time asked for this!

Works quickly.
There are problems with displaying on scales (the map view disappears) if several files are open.

Why is it not taken into account the zoom more than 18 (sasplanet)

for tests - SatImg WorldMap zoom 5-9 (sasPlanet)
https://yadi.sk/d/6lfjJOaA3SbFuR

[stelian]

FYI, compiling the lastest git master on recent linux is likely to fail because of an incompatibility between the version of libsqlite3-dev (>=3.19) and the usage of -DSQLITE_USE_LEGACY_STRUCT in OpenCPN.

While waiting for a proper fix (detecting the system libsqlite3 version and adjusting the build flags accordingly), you can edit CMakeLists.txt and remove the

add_definitions(-DSQLITE_USE_LEGACY_STRUCT)

line

[BlackSea]

Hi Dave,

thx for this feature in OpenCPN.

I have tried latest Build from AppVoyer and I haven't luck

I have tried both schemes ( xyz and tms) of mbtiles. Neither nor seems doesn't work

11:16:41: Loading chart data for ..\..\Charts\mbtiles\world.mbtiles
11:16:41: Adding chart file: ..\..\Charts\mbtiles\world.mbtiles
11:16:45: Finished chart database Update

11:18:03: Loading chart data for ..\..\Charts\mbtiles\worldtms.mbtiles
11:18:03: Adding chart file: ..\..\Charts\mbtiles\worldtms.mbtiles
11:18:03: Finished chart database Update

Both files provides zooms 3,4,6,7

But I see only GSHHS Charts

If you want, I can provide both Files

Regards
BlackSea

[sailorF54]

Originally Posted by stelian

FYI, compiling the lastest git master on recent linux is likely to fail because of an incompatibility between the version of libsqlite3-dev (>=3.19) and the usage of -DSQLITE_USE_LEGACY_STRUCT in OpenCPN.

While waiting for a proper fix (detecting the system libsqlite3 version and adjusting the build flags accordingly), you can edit CMakeLists.txt and remove the

add_definitions(-DSQLITE_USE_LEGACY_STRUCT)

line

No luck with that fix (commenting out or deleting the line) , still

/home/pytheas/OpenCPN/src/SQLiteCpp/src/Backup.cpp:16:21: fatal error: sqlite3.h: Aucun fichier ou dossier de ce type
compilation terminated.
CMakeFiles/SQLiteCpp.dir/build.make:65 :  la recette pour la cible  « CMakeFiles/SQLiteCpp.dir/src/SQLiteCpp/src/Backup.cpp.o » a échouée
make[2]: *** [CMakeFiles/SQLiteCpp.dir/src/SQLiteCpp/src/Backup.cpp.o] Erreur 1
make[2] : on quitte le répertoire « /home/pytheas/OpenCPN/build »
CMakeFiles/Makefile2:203 : la recette pour la cible « CMakeFiles/SQLiteCpp.dir/all » a échouée
make[1]: *** [CMakeFiles/SQLiteCpp.dir/all] Erreur 2
make[1] : on quitte le répertoire « /home/pytheas/OpenCPN/build »
Makefile:152 : la recette pour la cible « all » a échouée
make: *** [all] Erreur 2

[stelian]

Originally Posted by sailorF54

No luck with that fix (commenting out ou deleting the line) , still
[CODE]/home/pytheas/OpenCPN/src/SQLiteCpp/src/Backup.cpp:16:21: fatal error: sqlite3.h: Aucun fichier ou dossier de ce type
compilation terminated.

Off course, you also need libsqlite3 development libraries installed:

apt-get install libsqlite3-dev

(on Debian/Ubuntu/Rasbian etc, adapt for other distributions)

[sailorF54]

A single Mbiles chart (zoom level 14 to 17) file displays fine with W10.
Trying to split the same zone in 4 different charts (zoom 14, 15...) with/without quilting creates havoc: not display, boat sent to 0,0 etc...
(The idea being to be able to zoom in /out on the various levels)

[sailorF54]

Originally Posted by stelian

Off course, you also need libsqlite3 development libraries installed:

apt-get install libsqlite3-dev

(on Debian/Ubuntu/Rasbian etc, adapt for other distributions)

but

In file included from /home/pytheas/OpenCPN/src/SQLiteCpp/src/Backup.cpp:16:0:
/usr/include/sqlite3.h:3455:20: error: conflicting declaration ‘typedef struct Mem sqlite3_value’
 typedef struct Mem sqlite3_value;
                    ^
In file included from /home/pytheas/OpenCPN/src/SQLiteCpp/include/SQLiteCpp/Backup.h:14:0,
                 from /home/pytheas/OpenCPN/src/SQLiteCpp/src/Backup.cpp:12:
/home/pytheas/OpenCPN/src/SQLiteCpp/include/SQLiteCpp/Database.h:23:30: note: previous declaration as ‘typedef struct sqlite3_value sqlite3_value’
 typedef struct sqlite3_value sqlite3_value;
                              ^
CMakeFiles/SQLiteCpp.dir/build.make:65 : la recette pour la cible « CMakeFiles/SQLiteCpp.dir/src/SQLiteCpp/src/Backup.cpp.o » a échouée

[cagney]

Originally Posted by stelian

FYI, compiling the lastest git master on recent linux is likely to fail because of an incompatibility between the version of libsqlite3-dev (>=3.19) and the usage of -DSQLITE_USE_LEGACY_STRUCT in OpenCPN.

While waiting for a proper fix (detecting the system libsqlite3 version and adjusting the build flags accordingly), you can edit CMakeLists.txt and remove the

add_definitions(-DSQLITE_USE_LEGACY_STRUCT)

line

On Ubuntu 16.04 it's alright to compile after installing "libsqlite3-dev".
The version is only 3.11.0, so it's old enough to work.

Thomas

[stelian]

Originally Posted by sailorF54

previous declaration as ‘typedef struct sqlite3_value sqlite3_value

François, read my reply #7 above...

[sailorF54]

Originally Posted by stelian

François, read my reply #7 above...

But I have done it...
(i.e. deleting the line)