RWS data to GPX (or Opencpn)

[George64]

Hello,

The Dutch Government of Infrastructure is updating a data file with buoy information. Attached the buoy file. Rename to .zip and unzip. A csv file will show up. A snapshot looks like this:

VAARWATER BENAM_COD BENAMING INBEDRIJF X_RD Y_RD OBJ_SOORT IALA_CAT N_WGS_gms E_WGS_gms N_WGS_gm E_WGS_gm Obj_vorm_c OBJ_VORM Obj_kleur_c OBJ_KLEUR Kleurpatr_c KLEURPATR V_TT_c TT_TOPTEK TT_kleur_c TT_KLEUR TT_pat_c TT_KLR_PAT Sign_kar_c SIGN_KAR Sign_gr_c SIGN_GROEP SIGN_PERIO RACON_CODE Licht_kl_c LICHT_KLR OPGEHEVEN X_WGS84 Y_WGS84
AMER VW-AM -0103 A 5 01.01.2005 0 0 LK2600 MM. 2 51.43.30.35 004.44.57.49 51.43.5058 004.44.9582 1 spits 4 Groen # Niet toegewezen # Niet toegewezen # Niet toegewezen # Niet toegewezen 7 Iso (isophased) # Niet toegewezen 4 # 4 Groen # 4,749303 51,725097
AMER VW-AM -0104 A 7 01.01.2005 0 0 LK2600 MM. 2 51.43.23.20 004.45.42.10 51.43.3867 004.45.7017 1 spits 4 Groen # Niet toegewezen # Niet toegewezen # Niet toegewezen # Niet toegewezen 7 Iso (isophased) # Niet toegewezen 8 # 4 Groen # 4,761694 51,723111
AMSTERDAM-RIJNKANAAL VW-ARK -0101 AK 1-EH 2 01.01.2005 0 0 LK2600 MM. 3 52.22.21.74 004.57.17.07 52.22.3623 004.57.2845 3 bol 4,3 Groen/rood 1 Horizontaal 99 Kegel boven bol 4 Groen # Niet toegewezen 4 Q (quick-flashing) # Niet toegewezen # # 4 Groen # 4,954742 52,372706
AMSTERDAM-RIJNKANAAL VW-ARK -0102 AK 3 01.01.2005 485806,5125 125897,1832 LK1800 MM. 3 52.21.33.78 004.57.35.93 52.21.5630 004.57.5989 1 spits 4 Groen # Niet toegewezen # Niet toegewezen # Niet toegewezen # Niet toegewezen 3 LFl (long-flashing) # Niet toegewezen 5 # 4 Groen # 4,959981 52,359383
ANDELSE/AFGEDAMDE MAAS VW-AS -0101 AM 1 01.01.2005 0 0 TK1200.1 4 51.48.59.37 005.00.34.67 51.48.9895 005.00.5778 1 spits 4 Groen # Niet toegewezen 1 Kegel, punt naar boven 4 Groen # Niet toegewezen # Niet toegewezen # Niet toegewezen # # # Niet toegewezen # 5,009631 51,816492
ANDELSE/AFGEDAMDE MAAS VW-AS -0102 AM 2 01.01.2005 0 0 TK1200.2 4 51.49.03.10 005.00.35.96 51.49.0517 005.00.5993 2 stomp 3 Rood # Niet toegewezen 5 Cilinder 3 Rood # Niet toegewezen # Niet toegewezen # Niet toegewezen # # # Niet toegewezen # 5,009989 51,817528
ANDELSE/AFGEDAMDE MAAS VW-AS -0103 AM 3 01.01.2005 0 0 TK1200.1 4 51.48.46.69 005.00.36.05 51.48.7782 005.00.6008 1 spits 4 Groen # Niet toegewezen # Niet toegewezen # Niet toegewezen # Niet toegewezen # Niet toegewezen # Niet toegewezen # # # Niet toegewezen # 5,010014 51,812969

I think it would be handy if this information can be imported into Opencpn or any gps able to read gpx files. The file contains a lot of information. I'd like to know what info is needed to show a buoy with the proper information in Opencpn f.i.

Anyone who can help?

Many thanks in advance

[nohal]

George...
The file contains more than 9000 buoys, so converting it into a single GPX file would be kind of useless (even though possible as you can see on the attached screenshot).
Anyway, I wrote a script you can find at https://github.com/nohal/OpenCPNScri...s_buoys2gpx.sh to crunch the file and produce a GPX file for the data according to user supplied filter - while this was fun for me, creating partial GPX files that make sense and aren't too big wouldn't, so feel free to produce some - it should run fine on a Mac (from a Terminal of course).

Pavel

[George64]

Pavel,

Thanks for taking the time to write a script! Based on the csv file I can sort it by area and save that part of file for creating a gpx file. This would make it easier to load in Opencpn, I guess.

I'm not familiar with scripting :/ I run Opencpn on Windows. Is there a way to run this script on that pc as well?

Thanks again!

[nohal]

George...
Yes there is, but it's actually more work (and more intrusive to your Windows system) than installing Ubuntu or other Linux distro in a virtual machine and running the script there...

Pavel

[George64]

Thanks Pavel. I'll have a look what my options are under Windows 7 pro.

[George64]

Pavel....
Installed Ubuntu on Windows 7 using Virtualbox. Got your script running. I copy + paste the Betonning.csv file into the foleder where I also put the script. After a first run, using your example, I got:

./boeien.sh Betonning.csv VW-DD
<?xml version="1.0" encoding="utf-8" ?>
<gpx version="1.1" creator="OpenCPN" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.topografix.com/GPX/1/1" xmlns:gpxx="http://www.garmin.com/xmlschemas/GpxExtensions/v3" xsi:schemaLocation="http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd" xmlnspencpn="http://www.opencpn.org">
</gpx>

Actually there is no data shown. Am I overlooking something?

[nohal]

George...
Can we see Betonning.csv? Looks like none of the records contains "VW-DD"

Pavel

[George64]

Pavel...
thanks for your answer. Your question solved the problem......

George

[nohal]

George...
If you have the data divided in the source CSV files, just use "" (two doublequotes) as the second parameter. Also note that the CSV is expected to always have a header line, if it's not there you will loose the first buoy as no intelligence except throwing the first line away is in place...

Pavel

[George64]

I understand Pavel.

Kind of struggle to get the file from Ubuntu to Windows. Drag and drop does not work. Strange enough that works from windows to Ubuntu :/

[George64]

Pavel.....

Managed to get to Windows (better not start using O under Ubuntu )

One thing I'd like to ask, is it ok for you if I share the code on another forum?

[nohal]

George...
Help yourself, it's free as in either speech or beer... Just link the code from github, please - It will decrease the overall confusion in case I make some changes or extensions to it.

Pavel

[George64]

Pavel...
To me it would be disrespectful if I didn't ask you. I shared the link.

George

[rooiedirk]

Hoi everybody,
I made myself a script to convert the cvs file to a gpx.
For speed I did split it up in 3 files (I only did my own sailing area, but the script could be easely altered)
As symbols I am using the ones from Open Sea Maps.
Hope somebody finds this usefull
WKR
Dirk
PS the gpx files are drawn from the 20-6-13 cvs file

[rooiedirk]

oops,
forgot to add the UserIcon's