Cruisers Forum
 


Reply
 
Thread Tools Search this Thread Rate Thread Display Modes
Old 21-02-2018, 05:18   #61
Registered User

Join Date: Aug 2016
Posts: 149
Re: MBTiles for OpenCPN

Hi Pavel,

thx for your answer.
I have tried early morning latest build ( bdbcat has merged your PR)

I became an overzoomed layer, that appears if I zoom in/out.

Regards
BlackSea
Attached Thumbnails
Click image for larger version

Name:	mbtiles.jpg
Views:	146
Size:	362.8 KB
ID:	164526  
BlackSea is online now   Reply With Quote
Old 22-02-2018, 03:59   #62
Registered User

Join Date: Jun 2015
Posts: 379
Re: MBTiles for OpenCPN

Dave...
Quote:
Originally Posted by bdbcat View Post
did-g...


1. Please explain MBTiles overzoom issue again?

2. Sure, the CMake voodoo needs work. We plan to use system sqlite3 library on linux, and embedded zero-conf version on Windows, at least. On linux, we need to query the sqlite3 version, and do the right thing with incompatible typedefs.

3. Tell me more about advantage of git subtree. I took the easy (brute-force) method with a code dump, as the CMake rules found in the SQLiteCpp git repo tries to be a Swiss army knife, and had some troubles on Windows. I also hacked it a wee bit...
Send some code improvements, if you like.

O5 Plugin API:
As always, we make a serious effort not to break existing plugins with OCPN API updates, but sometimes wxWidgets changes make it unavoidable. Situation on O5 is not yet clear....

Thanks
Dave
1. It's not a mbtiles specific issue but with all these homebrew maps tools you can produce tiles with insane scales; scale loosely related, to say the least, with the actual one used for producing source data; compound with the 4x opencpn before overzoom alarm and you can end with inadequate charts.


3-
git subtree enable you to keep tracks of upstream projects, local patches, merge changes and so on, as you'd do with branches. Unlike git submodule they're transparent for other users of your git tree.

As usual it's a trade off. If you're sure you're never going to locally change them or upstream is all but unsupported then code dumps are fine, but predictions about the future are hard...

For examples:
* For coverity, travis , using git diff or git grep I'm keeping a O branch with some external plugins imported as subtrees, but not all compile properly so I do:

Importing :
Code:
git subtree add   -P plugins/foo --squash https://github.com/xxx/foo.git master
made some local changes to CMakeList.txt (often adding a BEFORE in INCLUDE_DIRECTORIES is enough) .

and from time to time
Code:
git subtree pull   -P plugins/foo --squash https://github.com/xxx/foo.git master
It's a normal pull, maybe with conflicts but it's rarely an issue.

* If the project is active and on github a workflow I like is:
- clone the repository
- make a working branch without all the stuff you don't want.
- pull this branch

Using a subtree for gdal with an intermediate O tree would be nice but merging back their and O changes is a lot of work and likely not worth it.

On O side MMEA code would be a good candidate for a subtree. There's at least 5 or 6 copy and paste versions of it with local changes and it's unclear how many should be backported.



BTW
Current mbtile code doesn't like crossing 180 longitude.
did-g is offline   Reply With Quote
Old 24-02-2018, 12:59   #63
Registered User

Join Date: Dec 2005
Location: Helsingborg
Boat: Dufour 35
Posts: 3,891
Re: MBTiles for OpenCPN

Quote:
Originally Posted by bdbcat View Post
Just when you thought it was safe to take a little break from OCPN.....

It is time to introduce MBTiles support in OpenCPN.
We are planning some level of support for MBTiles on OpenCPN Version 5.x, due out later this year. Full details are not yet determined.

To get started, we have prototyped native built-in support for MBTiles in the development tree. Source builders may find the code in github master branch now. Mac is untested so far. Windows users wanting to experiment may find a precompiled (but unsupported) Alpha version here:

https://www.dropbox.com/s/ljdtl742ux...setup.exe?dl=0

Notes:
1. OpenGL is required, at least for now. May be required for reasonable performance in production. We shall see.
2. MBTiles tilesets may be added to OCPN chart directories just like any other chart, in this case using the "mbtiles" file name extension.

So, where does one find MBTiles tilesets?
There are some precompiled tilesets floating around the internet. NOAA, for example, provides a download service for pre-built tilesets covering their range of RNCs. These are good tilesets, although quite large with many, many zoom levels included. There are others, of unknown provenance. Google around to find them, and decide for yourself if they are suitable for navigation.

On the other hand, many users will want to use some external tool to build tilesets for their navigational area of interest. Photo overlays come to mind here. I have experimented with SASPlanet, and find it quite useful for capturing tilesets of many popular and some more obscure cartographies. This Windows app downloads charts from many sources, and creates quilted tilesets directly from one user interface. Nice.

Our goal here:
Experimentation. What works, what can these things be used for, and how will this be productized for OCPN5.

As we move forward, we will talk more about the advantages and disadvantages of MBTiles. Those already having some experience with this chart format are encouraged to tell us what you know.

As always, I appreciate your continued support and feedback.
Dave
How accurate is the mbtiles format?
From https://github.com/mapbox/mbtiles-sp...er/1.1/spec.md
Quote:
The global-mercator (aka Spherical Mercator) profile is assumed
This is also called "Web Mercator".
NGA has this to say about the format: NGA: (U) NGA Advisory Notice on "Web Mercator" (UNCLASSIFIED)

Quote:
...
The NGA Geomatics Office has assessed the use of Web Mercator and other non-WGS 84 spatial reference systems may cause geo-location / geo-coordinate errors up to 40,000 meters. This erroneous geospatial positioning information poses an unacceptable risk to global safety of navigation activities,.....
At the same time NOAA is releasing their raster charts as mbtiles, but is that aimed for actual navigation or is it just a service to info apps? I have not found the answer yet.

So are mbtile good enough to use on OpenCPN?
Can they be used with the same confidence as BSB and S57 charts?

Thomas
cagney is offline   Reply With Quote
Old 24-02-2018, 13:18   #64
bcn
Registered User

Join Date: May 2011
Location: underway whenever possible
Boat: Rangeboat 39
Posts: 4,702
Re: MBTiles for OpenCPN

Thomas,

great heads-up.

Related to https://github.com/OpenCPN/OpenCPN/c...0ec97aa56e189e

boils down that the "Web Mercator (EPSG:3857) projection is a spherical projection and not a geoid based one - consequences laid down in the links Thomas found

Hubert
bcn is offline   Reply With Quote
Old 24-02-2018, 13:26   #65
Registered User

Join Date: Jan 2011
Location: France/UK
Boat: Gib'Sea 402
Posts: 546
Re: MBTiles for OpenCPN

Indeed - errors of up to 40kms could indeed be at least embarrassing ... ...
hoolie is offline   Reply With Quote
Old 24-02-2018, 13:31   #66
bcn
Registered User

Join Date: May 2011
Location: underway whenever possible
Boat: Rangeboat 39
Posts: 4,702
Re: MBTiles for OpenCPN

Now the question is:

how does that look like in the examples created via the different web services.
Which projection do they really use?

And checked against control points and well known objects or coast lines.

Nobody is forced to use EPSG:3857 for a mbTiles based data set. Who does, who not?
bcn is offline   Reply With Quote
Old 24-02-2018, 13:33   #67
Registered User
 
sailorF54's Avatar

Join Date: Dec 2009
Location: Perros-Guirec, France
Boat: Jeanneau Sunshine 36
Posts: 999
Re: MBTiles for OpenCPN

Quote:
Originally Posted by bdbcat View Post
SailorF54...

re:
Trying to split the same zone in 4 different charts (zoom 14, 15...) with/without quilting creates havoc: not display, boat sent to 0,0 etc...
(The idea being to be able to zoom in /out on the various levels)


Sample tiles showing this problem?

Thanks
Dave
I confirm my first findings.
When ingesting a large (a few 100 Mo) mbTiles chart (easy to generate with SASPlanet), or several charts adding up to the same amount, some kind of 'memory leak' destroys the vector chart config info, sends the boat position in the gulf of Guinea (0,0) etc...
Shutting down ans relaunching O usually clears the problem...
sailorF54 is offline   Reply With Quote
Old 24-02-2018, 15:54   #68
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 7,387
Re: MBTiles for OpenCPN

SailorF54...

More info, please...

OS?
What tile zoomScales are present in the loaded tilesets?
Does OCPN run OK, except for the noted problem? Or does it eventually hang/crash?

Thanks
Dave
bdbcat is offline   Reply With Quote
Old 24-02-2018, 16:09   #69
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 7,387
Re: MBTiles for OpenCPN

bcn...

re:
Nobody is forced to use EPSG:3857 for a mbTiles based data set. Who does, who not?

As far as I can see, there is no mechanism for using any projection other than WebMercator in mbtiles. The projection is implicit in the tile numbering scheme. There is no field in the metadata (if present at all) which specifies the projection. Indeed, from Mapbox spec:

Code:
Content

 The tiles table contains tiles and the values used to locate them.
 The zoom_level, tile_column, and tile_row columns MUST encode the location of the tile,
following the Tile Map Service Specification, with the restriction that the 
global-mercator (aka Spherical Mercator) profile MUST be used.
But there are doubtless mbtile servers in the wild that do not completely honor the mapbox specification. How can one know?

Regarding the US DOD proscription on WebMercator use:
I expect the 40km error factor quoted happens at extreme high latitude, where any Mercator-like projection suffers from numerical errors.

Food for thought.

Dave
bdbcat is offline   Reply With Quote
Old 24-02-2018, 18:35   #70
Registered User

Join Date: Dec 2005
Location: Helsingborg
Boat: Dufour 35
Posts: 3,891
Re: MBTiles for OpenCPN

I have compared US charts to mbtiles downloaded from SAS/Bing in a few spots, and can't find any greater discrepancies.
I also had a look at the South Coast of Britain. In NGA: (U) NGA Advisory Notice on "Web Mercator" (UNCLASSIFIED) there is a map of Britain showing that WebMerc is way out.

Click image for larger version

Name:	Screenshot_20180224_210043.png
Views:	81
Size:	154.4 KB
ID:	164766

Comparing SAS/Bing to UKHO raster charts and UKHO oeSENC charts I can see a difference of around 10 - 15 m, well within the error margin of the charts combined with my "test" method. Certainly nothing like the error that the NGA document suggests.
Click image for larger version

Name:	Screenshot_20180224_211720.png
Views:	79
Size:	52.3 KB
ID:	164771
Click image for larger version

Name:	Screenshot_20180224_212301.png
Views:	75
Size:	137.9 KB
ID:	164772

So what's going on?
Is WebMerc OK as long as we are using relatively small areas? For example not using one WebMerc chart of the whole of Britain.
Does NGA's 40 km error really refer to surface navigation?
Can someone test for possible errors in Norwegian Spitsbergen/Svalbard, about as close to a pole that we can use OpenCPN.

Thomas
cagney is offline   Reply With Quote
Old 24-02-2018, 19:39   #71
Registered User

Join Date: Feb 2010
Location: Tierra del Fuego
Boat: Phantom 19
Posts: 6,194
Re: MBTiles for OpenCPN

I have sure never seen errors in the range mentioned with correctly calibrated tilesets, but yes, it is theoretically possible. But in reality, Google Maps and similar services, which AFAIK use the spherical mercator projection, are not significantly less accurate than the official charts. In a lot of places it actually is the other way around as they do not rely on hundred years old surveys and simply believe the GPS on that plane taking the photos last year was correct. Which it usually seems to have been.

What I have seen so far:
1) Errors from converting and recalibrating non-mercator chart material to MBTiles using spherical mercator. This is a non-trivial task. Projections like transverse mercator etc. do exist for a reason and from what I see on the chartsets I have gathered so far not everybody realizes you don't make them spherical mercator or any other projection by establishing the 4 points in the corners. But this is a general problem seen even with KAPs, not specific to MBTiles.

2) The error differs between zoomlevels, especially when they are generated from different source charts (See the world basechart tileset posted by BlackSea earlier in this thread). This is a MBtiles specific phenomenon, possibly very dangerous and not always trivial to discover and much harder to see than with any other chart type. (See the attached screenshots with live AIS info from the SF bay overlaid on the forementioned tileset at two very near zoom levels - it of course does not make sense to measure the error at this scale, but the difference is very clearly visible)
We should probably make it possible to either hide certain zoomlevel or introduce a per-zoomlevel correction similar to what is now possible per-cell for CM93.

3) Complete bogus. As with KAPs produced by people who have no idea what they are doing.

Pavel
Attached Thumbnails
Click image for larger version

Name:	Screen Shot 2018-02-25 at 00.24.21.png
Views:	66
Size:	43.7 KB
ID:	164797   Click image for larger version

Name:	Screen Shot 2018-02-25 at 00.24.04.png
Views:	74
Size:	46.4 KB
ID:	164798  

nohal is online now   Reply With Quote
Old 24-02-2018, 21:51   #72
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 7,387
Re: MBTiles for OpenCPN

Pavel...

Looking at item (2)...

I studied the world.mbtiles. Turn on OCPN lat/lon grid, and look at how some latitude lines line up with various easily identified geo features. Or drop marks on spots like New York City, and zoom in/out.

Looks to me that one of the zoom levels (4?) is just plain wrong. I think this tile set is a GDAL re-projection of some other dataset. I don't know how GDAL does re-projection of raster data.

I doubt this is an inherent WebMercator projection problem, since it is clearly worse at one zoom level. More likely a problem in preparation of the tileset.

Which begs the rhetorical question: OCPN cannot evaluate the accuracy of an arbitrary raster image of claimed projection. This is a very difficult computational problem, in general. So how is a user to determine whether a tileset (obtained from whatever source) is actually usable for navigation?

Discuss?

Dave
bdbcat is offline   Reply With Quote
Old 24-02-2018, 23:00   #73
Registered User
 
sailorF54's Avatar

Join Date: Dec 2009
Location: Perros-Guirec, France
Boat: Jeanneau Sunshine 36
Posts: 999
Re: MBTiles for OpenCPN

Quote:
Originally Posted by bdbcat View Post
SailorF54...

More info, please...

OS?
What tile zoomScales are present in the loaded tilesets?
Does OCPN run OK, except for the noted problem? Or does it eventually hang/crash?

Thanks
Dave
Hello Dave

W10 (uptodate 1709)
OpenGL on
Zoom scale 12/14 to 17
When the problem occurs, no other chart set can be displayed. Closes OK when clicking x. Starts OK, all charts OK except for the changes noted above...
Regards

PS I cannot reproduce with O 4.99 built this morning (Sunday 0700 CET)
sailorF54 is offline   Reply With Quote
Old 25-02-2018, 05:48   #74
Registered User

Join Date: Feb 2010
Location: Tierra del Fuego
Boat: Phantom 19
Posts: 6,194
Re: MBTiles for OpenCPN

Quote:
Originally Posted by bdbcat View Post
Pavel...

Looking at item (2)...

I studied the world.mbtiles. Turn on OCPN lat/lon grid, and look at how some latitude lines line up with various easily identified geo features. Or drop marks on spots like New York City, and zoom in/out.

Looks to me that one of the zoom levels (4?) is just plain wrong. I think this tile set is a GDAL re-projection of some other dataset. I don't know how GDAL does re-projection of raster data.

I doubt this is an inherent WebMercator projection problem, since it is clearly worse at one zoom level. More likely a problem in preparation of the tileset.

Which begs the rhetorical question: OCPN cannot evaluate the accuracy of an arbitrary raster image of claimed projection. This is a very difficult computational problem, in general. So how is a user to determine whether a tileset (obtained from whatever source) is actually usable for navigation?

Discuss?

Dave
Dave...

Indeed, this is a problem of the respective tileset, not the way we read it or web-mercator itself. And I really don't see a way for us to solve it reading and displaying the tileset as there simply isn't enough information available to do it - pretty much everything has to be assumed to be "correct" by it's definition.

With eg. KAPs, there is more info available from which we can guess the calibration of a particular chart is of low quality or even if it technically isn't that bad, it is pretty well visible to the user that the chart changed and sanity has to be used. Neither is the case with the tiles.
To make it some more apparent, we might perhaps show the current zoomlevel integrated in the piano button?
And to somewhat resolve it the only option probably would be what I already wrote - per zoomlevel corrections/disabling.

And for the user the only suggestion as usually is - check you cartography against known good reference data. But this is not MBTiles specific neither, just even more important with it.

Pavel
nohal is online now   Reply With Quote
Old 25-02-2018, 11:09   #75
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 17,523
Images: 2
Re: MBTiles for OpenCPN

I remain pretty much in awe of this community. How do you guys know so much about everything it seems, even mbtiles. It is incredible.
rgleason is offline   Reply With Quote
Reply

Tags
enc, opencpn

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
mbtiles in opencpn seandepagnier OpenCPN 15 02-10-2017 01:36
MBTILES for opencpn ploubaz22 OpenCPN 4 25-12-2016 06:50
Mbtiles edelvoilier OpenCPN 3 23-05-2016 13:02
Virtual OpenCPN - 'OpenCPN on a Stick' r.fairman OpenCPN 23 16-10-2011 19:51

Advertise Here


All times are GMT -7. The time now is 03:59.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2024, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.