GRIB2 PlugIn

[did-g]

Hi,

Originally Posted by Rasbats

I have been comparing tidal current data from two sets of data using the Grib2 plugin:

1. ibi.nc
2. ucurr.grib/vcurr.grib made by CDO (Climate Data Objects) from ibi.nc

ibi.nc was downloaded from Copernicus.


Also interesting that the "hours since" shown for the file (at top of the screen) is the start date/time (2011-03-31 00:00:00) + 1 hour.

Mike

I still have to double check Dave's version but there's problem with wx and old dates some are off with gmt/local time. I had to use libc gmttime but it's obviously not portable, I'll try to find some time and look at it (biz taxes season here...).

[did-g]

Originally Posted by Ptizef

Hi Dave

I have downloaded and installed your 4.1 Windows build

I have very few grib2 files to test
- One from french "AROME" model .
https://donneespubliques.meteofrance...id_rubrique=51
Is named with ".grib2" termination . So it don't shows up in the file selector. I renamed it with ".grb" termination. Then I get "no valid data"
- One from https://meteo.gc.ca/grib/grib2_ens_naefs_f.html
Is also named with .grib2" termination.
Got "no valid data" when renamed it like above
- One from ftp://ftp.bsh.de/Stroemungsvorhersagen/
is named with ".grb2" termination so I add to rename it
Then the current data are shown but only at some zoom level

Tried to built with VS2010, but as you said it don't work
Got errors messages (and lot of warnings) (see thread grib2 branch)

Regards
Jean Pierre

Which one from meteofrance? IIRC only SP1 have useful data for ocpn.

for bsh IIRC the source grid is likely not regular and there's a lot of zero bins (no data for half of them) in the grib file, it doesn't play well with ocpn interpolation.

[Ptizef]

[QUOTE=did-g;2064906]Which one from meteofrance?
The only grib files I know from meteofance are from this site:
https://donneespubliques.meteofrance...id_rubrique=51
You can get them from two models : AROME and ARPEGE but I don't know
which data are inside.
Have you another solution to get meteofrance grib files ?
I have tested these grib2 on Linux (Kubuntu 15.10)
1) Built OK but with an error message :
||=== Build: grib_pi in OpenCPN (compiler: GNU GCC Compiler) ===|
/home/jeanpierre/Documents/OpenCPN/plugins/grib_pi/src/jasper/base/jas_stream.c|368|AVERTISSEMENT: the use of `tmpnam' is dangerous, better use `mkstemp'|
||=== Build finished: 1 error(s), 0 warning(s) (0 minute(s), 15 second(s)) ===|
2) tested the grib2 files from meteofrance (as above)
same result as with the windows version :
I get the message from GribUIDialog.cpp::424 "contains no valid data" which
certainly means the header has been read but not the data
3) I tested the grb2 files from ftp://ftp.bsh.de/Stroemungsvorhersagen/
These files have only current data, and one day by file
I was able to load 3 files (3days). the direction arrows are shown at
certain zoom level and disappear at other. Also the data are not shown in the
data at cursor dialog (see two screen shot)

Jean Pierre

[Ptizef]

I forgot:
grib1 files seem to work fine
JP

[rgleason]

Originally Posted by maitai

zyGrib does not read grib2 format, qtVlm does. We have similar problems extracting a valid grib1 or grib2 file from myOcean data. Still working on that.

So maybe we could use qtvlm grib download or launcher_pi and degrib command line to download an area with multiple types of data across a small time frame for testing, from Ncdc

[rgleason]

I mean use these links and the degrib command line to join the data into one grib2 file. Will work on this later.

Leads to
Index of /ensemble/naefs/grib2/raw

Picked one path
Index of /ensemble/naefs/grib2/raw/00/000

Downloaded each type and tested. Not many work but some do.

[did-g]

Originally Posted by Ptizef

Which one from meteofrance?
The only grib files I know from meteofance are from this site:
https://donneespubliques.meteofrance...id_rubrique=51
You can get them from two models : AROME and ARPEGE

I'm selecting SP1 'sous paquet'
for example
https://donneespubliques.meteofrance...id_rubrique=51

cf screen capture

Originally Posted by Ptizef

but I don't know which data are inside.

List is in
https://donneespubliques.meteofrance...er2016_198.pdf
On linux you can also use grib_ls

[rgleason]

Yes did-g, that works. Thanks for giving me a shove in the right direction.
Have downloaded that Arome. Will see what it has.

[Rasbats]

Also downloaded ... 40mb!

Check boxes for Wind and Pressure do not seem to work.

Isobars in some areas do not seem very smooth?

Mike

[rgleason]

When I first opened it it choked and aeemed incompatible. Then opened up again and it worked with most of the data-types. More later.

[bdbcat]

Rasbats...

re:
I have been comparing tidal current data from two sets of data using the Grib2 plugin:


Lets break this down a little...

1. Do you think the BSH current data decode/display by GRIB2 is accurate?
If so, lets consider this the "reference" data.

2. Please tell us how you have processed the ibi.nc data to produce a GRIB2 theoretically equivalent to the BSH file.

3. I understand that the time reference of the ibi.nc conversion is incorrect. Please amplify.

4. Can you post some small GRIB2 files here for analysis?

My objective here is to determine whether OCPN has a problem, or CDO has a problem. Or both.

Thanks
Dave

[bdbcat]

Folks....

Where to get GRIB2 files for testing...

1. According to this post, there seems to be some trouble with GRIB2 files made by conversion from other sources. (e.g. ibi.nc to GRIB2 using CDO).
http://www.cruisersforum.com/forums/...ml#post2063968

What is needed here is to discover whether the conversion process is in error, or the OCPN GRIB2 decoding is faulty, or both. The way to determine that is by comparing OCPN display against another well known GRIB2 viewer, and looking for differences. Frankly, there are very few simple GRIB2 viewers available online. The best seem to be on Android...

You may also find the wgrib2 utility to be useful.
Climate Prediction Center - wgrib2: grib2 utility

We should focus on determining if the OCPN GRIB2 decode/display is accurate and reliable for a properly formed GRIB2 file. Lets leave the "how do I convert/manipulate GRIB2 files?" workflow questions for another thread.

2. did-g has shown us a way to download filtered GRIB2 files from the French Arome model. As has been noted, the full model available by online download is huge, and so not of much practical use "in-the-raw" for actual navigation. For practical testing, take a look at this post, and give it a try.
http://www.cruisersforum.com/forums/...ml#post1910594

As an example, the following line:
curl -o today.grb --compressed "http://195.154.231.142/grib/arome?x=-2X=2&y=47&Y=48&r=w"

(36 hour wind forecast by hour in a quite small area) gave a file size of 1.2 Mb.

I find that the current OCPN GRIB2 decodes and displays correctly all of the parameters available by this server.
e.g. current,wind 10m,gust,temp 2m,pressure,rain,cloud cover

This suggests that at least the minimum standard GRIB2 variables available from the AROME model are compatible with OCPN GRIB2.

3. For NOAA GFS model forecasts, I find it best to use the grib_filter interface provided here:
NOMADS-NOAA Operational Model Archive and Distribution System

Choose the model you want, select the "grib_filter" link, and then choose the variables, timescale, and geographical area you want downloaded. This interface produces very small file download sizes, so is quite useful for testing various options of GRIB2.

Have fun...
Dave

[Sailor_Hutch]

Originally Posted by bdbcat

Folks....

...
What is needed here is to discover whether the conversion process is in error, or the OCPN GRIB2 decoding is faulty, or both. The way to determine that is by comparing OCPN display against another well known GRIB2 viewer, and looking for differences. Frankly, there are very few simple GRIB2 viewers available online. The best seem to be on Android...
...
Have fun...
Dave

Hi: I don't know if you've tried GrADS as a cross-check?


https://programmingmiscellany.wordpr...t-the-weather/

- Ron

[Sailor_Hutch]

The problem with the suggestion in my last post could be the projection? I think that might make it quite a bit more complicated than the graphics in the associated link would imply ...

[Rasbats]

Dave,

Did some comparisons of BSH data and tidal current from a Copernicus nc file for 08/1200UTC.

motu_client_script asked for Copernicus data starting 2016-03-08 12:00:00UTC. This produced ibi.nc.7z.

Current_no_2016 etc was the grb2 from BSH.

Copy.7z was the batch file to translate the nc to grib(2) via CDO. IBI_ucurr.grib was the result (IBI_vcurr.grib was also produced but I find they have the same data).

Screenshots show the comparison:
Current.png is BSH
ibi_vcurr.png is Copernicus via CDO
ncf_plugin.png is Copernicus via netCDF.lib api plugin

All for the same time.

With the CDO grib using Grib2 plugin you see 2011-03-31 01:00:00UTC in the title bar. With ncdump the start date/time is found to be 2011-03-31 00:00:00UTC.

Again with CDO you see the earliest forecast time available is 1400. This corresponds to data for 1200 from both the ncdf plugin and BSH current data.

Writing the ncdf plugin I found that wxTimeSpan caused me a problem with the date 2011-03-31 00:00:00UTC. Adding the time span for the forecast produced a date/time one hour earlier than actual (as verified by a Web calculator). Worked around this by adding one hour to the forecast time spans (toUTC did not help). 31 March 2011 was Daylight Saving Time in Europe. Is this significant?

CDO.exe was compiled for Windows here, with grib_api support built in.
Perhaps someone on Linux can try the same exercise. Does did-g have an example.

Will the CDO route ever be used? Could this problem be ignored?

However the good quality data from Copernicus should not be ignored.

Mike