gribs concatenation and grib version2

[did-g]

Hi,

I pushed a POC https://github.com/did-g/OpenCPN.git
in branch WIP_next

- It uses a multiple files style dialog box, use CTRL and SHIFT for selecting files.

It does interpolation so you can open a 3h wind GRIB with 1h current GRIB and get some display (interpolated values are often wrong and it's slow but good enough for now).

If you open files with the same record type at the same time it will do, something..., surely stupid.

- I chainsawed stuff from qtvlm and ucar.edu for grib2, it's linux only for now.
You have to install jasper for jpeg decoding, on debian like system:
apt-get install libjasper1
apt-get install libjasper-dev

It's slow and uses a lot of memory but it's a start, no waves or precipitation yet.

Without jasper but with libgrib-api-tools installed you can convert from jpeg to
grid_simple encoding with:

grib_set -s dataRepresentationTemplateNumber=0 src.grb dst.grb

And still open grib2 files, as a matter of fact you can convert grib2 to grib1 with
grib_set -s edition=1 src.grb dst.grb
but it's buggy and there's not always a match between v2 records type and v1 type.

If gribs are too big you can split them with:
grib_copy src.grb dst[shortName].grb
and then open only dts<whatever>.grb
with whatever:
10u and 10v for wind (need both)
ucurr and vcurr for current
gust for, well gust (not ugust or vgust, ocpn doesn't use them)
tcc: total cloud cover.
msl: pressure

and so on.



- If you use Sean's weather_routing you have to clone:
https://github.com/did-g/weather_routing_pi.git

Internal grib record format is different and incompatible with older weather_routing versions.

The textbook Manche (English Channel) crossing COG example. model MeteoFrance AROME 0.025° cells and MyOcean current grib. (brown arrows are wind over water, black wind arrows are current).

[seandepagnier]

Originally Posted by did-g

Hi,

I pushed a POC https://github.com/did-g/OpenCPN.git
in branch WIP_next

- It uses a multiple files style dialog box, use CTRL and SHIFT for selecting files.

It does interpolation so you can open a 3h wind GRIB with 1h current GRIB and get some display (interpolated values are often wrong and it's slow but good enough for now).

Why is it slow, and more important, why is it "often wrong" ?

If you open files with the same record type at the same time it will do, something..., surely stupid.

What to do in this case? Ask user which file to prefer, or?

- I chainsawed stuff from qtvlm and ucar.edu for grib2, it's linux only for now.
You have to install jasper for jpeg decoding, on debian like system:

Is there a good reason we can't just start linking with libgrib and use that? Maybe it's too slow for us.

- If you use Sean's weather_routing you have to clone:
https://github.com/did-g/weather_routing_pi.git

Internal grib record format is different and incompatible with older weather_routing versions.

I will look into this.

[rgleason]

Hey this is a start! Thanks for giving it a start. If we can just do a file concatination like file1 + file2 ^ then shouldn't it be pretty simple to do? Maybe libgrid is a good way to do it. Perhaps this should be part of the grib pluggin.

[rgleason]

Thanks did-g for the effort.

[did-g]

Originally Posted by boat_alexandra

Why is it slow, and more important, why is it "often wrong" ?

It interpolates every missing records.
When interpolating, I don't use the right time for the new record...
I'm pretty sure it's also using the wrong math for current.

Originally Posted by boat_alexandra

What to do in this case? Ask user which file to prefer, or?

I don't know.

Originally Posted by boat_alexandra

Is there a good reason we can't just start linking with libgrib and use that? Maybe it's too slow for us.

by libgrib do you mean the one used in qtvlm?
Yes it could be used, it seems to do job, anyway it's a small change.
Speed shouldn't matter, most of the time is spent in jpeg decoding or bunzip2

[fogmachine]

I wrote some Python to generate curl to pull GRIB2 direct from NOMADS and found a tool from UCAR to convert them to GRIB1 for OpenCPN and/or zyGrib. Works fine, no chainsaw required.

Getting there required many days of looking at tools including Jasper, the Java tools, installing Fortran on my Mac, etc. But in the end it was pretty straightforward.

[rgleason]

Why can't grib_pi be improved to support concatinatipn and grib2?

[bcn]

Météo-France GRIB files finally for free – Navigation Mac

Another reason to get Grib 2.0 running....

[rgleason]

Looks good to me. Will be better and finer data for France and Europe than Noaa. Will provide cuonterpoint comparison to noaa for the world. Both models will get better.

[did-g]

Pushed a new version.

For testing you can download french meteo (AROME 0.025 cells, 36 hours) and IBI (myocean 0.033 cell) current at:

http://195.154.231.142/grib/arome
eg:
curl -o today.grb --compressed "http://195.154.231.142/grib/arome?x=-8&X=2&y=47&Y=51&r=cwgtprn"

with:
x: long min (neg for west)
X: long max
y: lat min
Y: lat max

and r records
c current
w wind 10m
g gust
t temp 2m
p pressure
r rain
n cloud cover