OCN-VoyageDataRecorder fails to heip me supply OPENDEPTHproject with sounding data

[Montyfrisian]

Hi folks,


most of you will have heard of OPENSEAMAP ... Cool idea, isn't it, an open community that provides charts for free?
Except that the most important thing is missing: depth data.


Well, this is currently changing as OSM naval charts DO display depth data now, contours and 25m and other lines plus tracks of boats that have recorded their echo sounding data, and a very small group of developers are working hard to really get it going.



It seems the initiative is from some German people as most of the depth tracks can be found in the Western part of the Baltic, the home of most German boaters. If you go further afield, there is hardly any depth track to be found ...



Anyway, the idea is to assemble more and more skippers to hand in their random voyage data (or even recordings that have been privately undertaken with a purpose to sound their home water systematically) as to combine all that stuff to cover whole areas with up-to-date soundings. Highly useful what with the authorities acting under the lack of public funding and certainly highly useful in tidal areas where silting and sanding is frequent and the boating ground of recreational craft is of little interest for them.


Now, I, as a fan of that kind of community project am trying to contribute too. As a fan of OPEN!cpn I employed the VDR that comes in handy.


It's just that there is a problem (a bug?) somewhere. The VDR protocols the NMEA-data from the ships sensors and writes them into a simple txt file. On


https://depth.openseamap.org/#tracks


you can upload that file where it is identified, duely unzipped (unless it hadn't been zipped before), checked and processed in order to display your track recording along with the depth data on the chart. NMEA0183 is not the only format that is feasible, but it's the one I have rudimentary ideas about.


The problem is that there seems to be something wrong with the NMEA txt-file - I receive a notice that the format will not be recognised.


Is it possible that Ocpn, while producing files that are identifiable for Ocpn itself, screws up the format thus rendering it unreadable to any other machine? The protocol, I mean the "text" with the NMEA strings, looks fine to me ...



Anyone here with enough knowledge about NMEA0183, Ocpn, its VDR and hopefully some experience in uploading stuff to opendepth?



Cheers,
and a happy new year to all sailing and opendataprojects enthusiasts.



M from Germany


PS I may submit one of my files to be looked at but for the time being I refrain from producing too much digital junk

[nohal]

The VDR plugin does nothing with the NMEA data, just records what is being received on the input.
Of course without seeing what exactly you are submitting and what exactly you receive as an error when you do so (If anything), we are not able to help you.

[sv_pelagia]

OpenCPN VDR works well for our depth surveys.



I wrote a short script to take the VDR txt file and associate depths with lat & long and output gpx and csv files (corrected for tide level).



https://sailing-pelagia.blogspot.com...ou-go.html?m=0


There is also commercial software that does this and much more:
https://reefmaster.com.au/

[Montyfrisian]

Thanks for answering, nohal and sv_pelagia.


Nohal, should I upload one of the nmea files that I recorded with OpenCpn?? Pardon me if I am hesitant about that but I can't see why I should shoot many megabites of data around when it is simply the kind of thing that the VDR produces and leaves as it is.


I had a suspicion about the order of the nmea-data or that the VDR fails to produce a header or time stamp or anything compulsory but you suggest that there is no such thing.



And as to the specific problems of uploading stuff to the OPENDEPTH-project ... Well, I simply followed the instructions (https://depth.openseamap.org/#instructions), putting together the setup information - size of the ship, echosounder position in the boat etc and finally uploading the txt-file that the VDR produced.



That project enables users of different kinds of potters / data formats such as NMEA2000 and others to forward the data collections.


I thought that using NMEA-0183 was the most straightforward way to do submit data, and so I am absolutely puzzled why it does not work.

[nohal]

The reason is VDR does not produce anything, it just saves what your system produces to a file.
I have no idea what your system produces, nor does anybody else, so neither me, nor anybody else can help you unless you show use that data.

You neither answered my second question: What exact error did you get after uploading your data, if any?

Sorry, but insisting will not get you anywhere, providing the information we ask for might.

[sv_pelagia]

Originally Posted by Montyfrisian

I had a suspicion about the order of the nmea-data or that the VDR fails to produce a header or time stamp or anything compulsory but you suggest that there is no such thing.

If your NMEA0183 datastream (which is saved by VDR) includes GPS output, the date(s) and time(s) UTC will be in the VDR.



For your own info, it helps to name/rename the resulting txt file with an informative name.

[Montyfrisian]

I attach one of the files that was created by the VDR.
Tricky though, because I had to delete most of it to make it uploadable. There is depth data in it but I doubt this will really help to do a proper check of the file ...
(Is there another way of uploading the complete txt-file whose size is 17.1Mb?)



Before I uploaded the data I checked for myself if there was any depth data in it. Of course it was, otherwise a replay within OpenCpn would not have worked ...


Another thing I tried was to play around with re-naming the file, to make it identifiable and to make sure syntax was right (like: .... .txt or TXT or omitting the suffix altogether.


I also submitted one of the two said files to the guys at the project to have a look at it from their perspective. There was feedback tonight from the coordinator that a data specialist will take a look soon.


Additionally, I attach a screenshot of the place in OPENDepth where one has to upload stuff. This is a somewhat helpless gesture, SORRY!, but for the time being this is the only thing that I am able to do to localise/describe the area where the problems arise. In my former posts I quoted the links to those places. I suppose the only way to understand the "exact error" is to log in there and have a go with a VDR-txt-file.


Taking in the information from the screenshot you can see that uploading took place, alright, but the green boxes with the question marks show that the file type has not been recognised. I don't know it that refers to .txt or to the NMEA-0183 format. Anyway, if things work out alright, I am told, the boxes should be red, displaying "NMEA"
Compression is not a must, I am told, and as I wrote before I tried to upload the files both zipped and in non-zipped form. Both my tracks are uninterrupted (continuous) though beginning and ending at random, but that cannot really matter much, can it?


Lastly, I attach a screenshot of the window where anything uploaded should show up after one or two hours. The tracks displayed appear colour-coded in the normal OPENSEAMAP chart. However, if you use the "TRACK VIEWER" you can zoom up and thus get to the level where the concrete sounding data is displayed.

[nohal]

In the sample file you posted there are no position data at all, a depth without a position is of course not usable for a project like this. Perhaps you were just too brutal while shortening it though as it is really minimal.

[nohal]

The way to share a large file is to upload it anywhere (Dropbox, OneDrive, Google Disk, or anything similar) and post a link

[sv_pelagia]

I'm surpised that little 15-line txt file was 412 bytes! I have1000-line VDR txt files that are smaller.

[sv_pelagia]

I'm surpised that little 17-line txt file was 412 bytes! I have >1000-line VDR txt files that are smaller.

[nohal]

Originally Posted by sv_pelagia

I'm surpised that little 17-line txt file was 412 bytes! I have >1000-line VDR txt files that are smaller.

That would surprise me on the other hand as a minimal NME0183 sentence is a lead in character, 2 byte talker ID, 3 byte sentence ID, asterisk, 2 byte checksum and a newline delimiter, which is 10 bytes per line minimum even if the sentence would have zero fields and contained no information (OK, 7 if the optional checksum is not there).
Compressed, maybe, but sure not raw.

[Montyfrisian]

I might try copying some more sentences into a doc-file, but before I do that I'd like to share that I may have stumbled upon a few hints as to what could be wrong with the data.


I rummaged around and came across a site that's called nmeagen.org that uses OSM maps. Now, interestingly, I was able to upload one of my files successfully - the short track of about 5 nm showed up on the map (till one of my crew ripped off the power cable). I could not upload the other, which contained even more data such as AIS.


On uploading the files a feedback window appears informing you about GPS frequency, valid and invalid sentences and "points", whatever that may be.


Now both files lacked the frequency input, but that did not matter. Both contained invalid sentences, which obviously did not matter much either, but:


one files had these "points" while the other, more extensive file, did not.


The one where the "points" could be identified, would be uploaded. In the case of the other nothing happened.


I attach screenshots which will hopefully give a clue to you clever tekkie people.

[Montyfrisian]

Ok, I attach a doc-file here into which I copied a larger bulk of data. This time there is GPS sentences in it too (but I just took a very quick glance to check, so hopefully that's ok)

[nohal]

OK, this file looks better. I do not see anything obviously preventing the data to be used for the purpose.

I have looked at their code, but it looks like being unmaintained for 5+ years and can't be even built because the repository with the dependencies that they claim to be available on the website doesn't exist. As a result I can't try to process the logs locally using their software.

The binaries they claim to be available on the website are not there neither.

I have also created an account there and tried uploading something. As nothing happens for several hours, my suspicion is their backend processing is not working. But there is no way for me to confirm or deny this.

I am afraid the options we have are exhausted.