Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Rate Thread Display Modes
Old 06-03-2016, 11:47   #31
Registered User

Join Date: Jun 2015
Posts: 202
Re: GRIB2 PlugIn

Hi,
Quote:
Originally Posted by Rasbats View Post
I have been comparing tidal current data from two sets of data using the Grib2 plugin:

1. ibi.nc
2. ucurr.grib/vcurr.grib made by CDO (Climate Data Objects) from ibi.nc

ibi.nc was downloaded from Copernicus.


Also interesting that the "hours since" shown for the file (at top of the screen) is the start date/time (2011-03-31 00:00:00) + 1 hour.

Mike
I still have to double check Dave's version but there's problem with wx and old dates some are off with gmt/local time. I had to use libc gmttime but it's obviously not portable, I'll try to find some time and look at it (biz taxes season here...).
__________________

__________________
did-g is offline   Reply With Quote
Old 06-03-2016, 12:07   #32
Registered User

Join Date: Jun 2015
Posts: 202
Re: GRIB2 PlugIn

Quote:
Originally Posted by Ptizef View Post
Hi Dave

I have downloaded and installed your 4.1 Windows build

I have very few grib2 files to test
- One from french "AROME" model .
https://donneespubliques.meteofrance...id_rubrique=51
Is named with ".grib2" termination . So it don't shows up in the file selector. I renamed it with ".grb" termination. Then I get "no valid data"
- One from https://meteo.gc.ca/grib/grib2_ens_naefs_f.html
Is also named with .grib2" termination.
Got "no valid data" when renamed it like above
- One from ftp://ftp.bsh.de/Stroemungsvorhersagen/
is named with ".grb2" termination so I add to rename it
Then the current data are shown but only at some zoom level

Tried to built with VS2010, but as you said it don't work
Got errors messages (and lot of warnings) (see thread grib2 branch)

Regards
Jean Pierre
Which one from meteofrance? IIRC only SP1 have useful data for ocpn.

for bsh IIRC the source grid is likely not regular and there's a lot of zero bins (no data for half of them) in the grib file, it doesn't play well with ocpn interpolation.
__________________

__________________
did-g is offline   Reply With Quote
Old 06-03-2016, 13:03   #33
Registered User

Join Date: Sep 2009
Location: Angers - France
Boat: Beneteau First 29 Ptizef
Posts: 743
Re: GRIB2 PlugIn

[QUOTE=did-g;2064906]Which one from meteofrance?
The only grib files I know from meteofance are from this site:
https://donneespubliques.meteofrance...id_rubrique=51
You can get them from two models : AROME and ARPEGE but I don't know
which data are inside.
Have you another solution to get meteofrance grib files ?
I have tested these grib2 on Linux (Kubuntu 15.10)
1) Built OK but with an error message :
||=== Build: grib_pi in OpenCPN (compiler: GNU GCC Compiler) ===|
/home/jeanpierre/Documents/OpenCPN/plugins/grib_pi/src/jasper/base/jas_stream.c|368|AVERTISSEMENT: the use of `tmpnam' is dangerous, better use `mkstemp'|
||=== Build finished: 1 error(s), 0 warning(s) (0 minute(s), 15 second(s)) ===|

2) tested the grib2 files from meteofrance (as above)
same result as with the windows version :
I get the message from GribUIDialog.cpp::424 "contains no valid data" which
certainly means the header has been read but not the data
3) I tested the grb2 files from ftp://ftp.bsh.de/Stroemungsvorhersagen/
These files have only current data, and one day by file
I was able to load 3 files (3days). the direction arrows are shown at
certain zoom level and disappear at other. Also the data are not shown in the
data at cursor dialog (see two screen shot)

Jean Pierre
Attached Thumbnails
Click image for larger version

Name:	capture1.png
Views:	75
Size:	158.4 KB
ID:	120110   Click image for larger version

Name:	capture2.png
Views:	50
Size:	193.2 KB
ID:	120111  

__________________
Ptizef is offline   Reply With Quote
Old 06-03-2016, 13:09   #34
Registered User

Join Date: Sep 2009
Location: Angers - France
Boat: Beneteau First 29 Ptizef
Posts: 743
Re: GRIB2 PlugIn

I forgot:
grib1 files seem to work fine
JP
__________________
Ptizef is offline   Reply With Quote
Old 06-03-2016, 15:59   #35
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,398
Re: GRIB2 PlugIn

Quote:
Originally Posted by maitai View Post
zyGrib does not read grib2 format, qtVlm does. We have similar problems extracting a valid grib1 or grib2 file from myOcean data. Still working on that.
So maybe we could use qtvlm grib download or launcher_pi and degrib command line to download an area with multiple types of data across a small time frame for testing, from Ncdc
__________________
rgleason is offline   Reply With Quote
Old 06-03-2016, 16:06   #36
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,398
Re: GRIB2 PlugIn

I mean use these links and the degrib command line to join the data into one grib2 file. Will work on this later.
Quote:
Leads to
Index of /ensemble/naefs/grib2/raw

Picked one path
Index of /ensemble/naefs/grib2/raw/00/000

Downloaded each type and tested. Not many work but some do.
__________________
rgleason is offline   Reply With Quote
Old 07-03-2016, 13:29   #37
Registered User

Join Date: Jun 2015
Posts: 202
Re: GRIB2 PlugIn

Quote:
Originally Posted by Ptizef View Post
Which one from meteofrance?
The only grib files I know from meteofance are from this site:
https://donneespubliques.meteofrance...id_rubrique=51
You can get them from two models : AROME and ARPEGE
I'm selecting SP1 'sous paquet'
for example
https://donneespubliques.meteofrance...id_rubrique=51

cf screen capture

Quote:
Originally Posted by Ptizef View Post
but I don't know which data are inside.
List is in
https://donneespubliques.meteofrance...er2016_198.pdf
On linux you can also use grib_ls
Attached Thumbnails
Click image for larger version

Name:	Capture du 2016-03-07 21:18:04.png
Views:	55
Size:	184.4 KB
ID:	120189  
__________________
did-g is offline   Reply With Quote
Old 07-03-2016, 13:37   #38
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,398
Re: GRIB2 PlugIn

Yes did-g, that works. Thanks for giving me a shove in the right direction.
Have downloaded that Arome. Will see what it has.
__________________
rgleason is offline   Reply With Quote
Old 07-03-2016, 14:11   #39
Registered User

Join Date: May 2012
Posts: 408
Re: GRIB2 PlugIn

Also downloaded ... 40mb!

Check boxes for Wind and Pressure do not seem to work.

Isobars in some areas do not seem very smooth?

Mike
__________________
Rasbats is online now   Reply With Quote
Old 07-03-2016, 15:49   #40
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,398
Re: GRIB2 PlugIn

When I first opened it it choked and aeemed incompatible. Then opened up again and it worked with most of the data-types. More later.
__________________
rgleason is offline   Reply With Quote
Old 07-03-2016, 19:37   #41
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 4,884
Re: GRIB2 PlugIn

Rasbats...

re:
I have been comparing tidal current data from two sets of data using the Grib2 plugin:


Lets break this down a little...

1. Do you think the BSH current data decode/display by GRIB2 is accurate?
If so, lets consider this the "reference" data.

2. Please tell us how you have processed the ibi.nc data to produce a GRIB2 theoretically equivalent to the BSH file.

3. I understand that the time reference of the ibi.nc conversion is incorrect. Please amplify.

4. Can you post some small GRIB2 files here for analysis?

My objective here is to determine whether OCPN has a problem, or CDO has a problem. Or both.

Thanks
Dave
__________________
bdbcat is offline   Reply With Quote
Old 07-03-2016, 19:45   #42
Marine Service Provider
 
bdbcat's Avatar

Join Date: Mar 2008
Posts: 4,884
Re: GRIB2 PlugIn

Folks....

Where to get GRIB2 files for testing...

1. According to this post, there seems to be some trouble with GRIB2 files made by conversion from other sources. (e.g. ibi.nc to GRIB2 using CDO).
GRIB2 PlugIn

What is needed here is to discover whether the conversion process is in error, or the OCPN GRIB2 decoding is faulty, or both. The way to determine that is by comparing OCPN display against another well known GRIB2 viewer, and looking for differences. Frankly, there are very few simple GRIB2 viewers available online. The best seem to be on Android...

You may also find the wgrib2 utility to be useful.
Climate Prediction Center - wgrib2: grib2 utility

We should focus on determining if the OCPN GRIB2 decode/display is accurate and reliable for a properly formed GRIB2 file. Lets leave the "how do I convert/manipulate GRIB2 files?" workflow questions for another thread.

2. did-g has shown us a way to download filtered GRIB2 files from the French Arome model. As has been noted, the full model available by online download is huge, and so not of much practical use "in-the-raw" for actual navigation. For practical testing, take a look at this post, and give it a try.
gribs concatenation and grib version2

As an example, the following line:
curl -o today.grb --compressed "http://195.154.231.142/grib/arome?x=-2X=2&y=47&Y=48&r=w"

(36 hour wind forecast by hour in a quite small area) gave a file size of 1.2 Mb.

I find that the current OCPN GRIB2 decodes and displays correctly all of the parameters available by this server.
e.g. current,wind 10m,gust,temp 2m,pressure,rain,cloud cover

This suggests that at least the minimum standard GRIB2 variables available from the AROME model are compatible with OCPN GRIB2.

3. For NOAA GFS model forecasts, I find it best to use the grib_filter interface provided here:
NOMADS-NOAA Operational Model Archive and Distribution System

Choose the model you want, select the "grib_filter" link, and then choose the variables, timescale, and geographical area you want downloaded. This interface produces very small file download sizes, so is quite useful for testing various options of GRIB2.

Have fun...
Dave
__________________
bdbcat is offline   Reply With Quote
Old 08-03-2016, 07:26   #43
Registered User
 
Sailor_Hutch's Avatar

Join Date: Nov 2012
Location: Dreaming - through the bars to the Chesapeake... Land cabin: near Raleigh, North Carolina
Posts: 461
Re: GRIB2 PlugIn

Quote:
Originally Posted by bdbcat View Post
Folks....

...
What is needed here is to discover whether the conversion process is in error, or the OCPN GRIB2 decoding is faulty, or both. The way to determine that is by comparing OCPN display against another well known GRIB2 viewer, and looking for differences. Frankly, there are very few simple GRIB2 viewers available online. The best seem to be on Android...
...
Have fun...
Dave
Hi: I don't know if you've tried GrADS as a cross-check?


https://programmingmiscellany.wordpr...t-the-weather/

- Ron
__________________
Sailor_Hutch was born for water. His 130 pounds, well insulated, floats like a bouy. With webbed paws, he gracefully paddles - The Umbrella Man.
Sailor_Hutch is offline   Reply With Quote
Old 08-03-2016, 07:34   #44
Registered User
 
Sailor_Hutch's Avatar

Join Date: Nov 2012
Location: Dreaming - through the bars to the Chesapeake... Land cabin: near Raleigh, North Carolina
Posts: 461
Re: GRIB2 PlugIn

The problem with the suggestion in my last post could be the projection? I think that might make it quite a bit more complicated than the graphics in the associated link would imply ...
__________________
Sailor_Hutch was born for water. His 130 pounds, well insulated, floats like a bouy. With webbed paws, he gracefully paddles - The Umbrella Man.
Sailor_Hutch is offline   Reply With Quote
Old 08-03-2016, 14:39   #45
Registered User

Join Date: May 2012
Posts: 408
Re: GRIB2 PlugIn

Dave,

Did some comparisons of BSH data and tidal current from a Copernicus nc file for 08/1200UTC.

motu_client_script asked for Copernicus data starting 2016-03-08 12:00:00UTC. This produced ibi.nc.7z.

Current_no_2016 etc was the grb2 from BSH.

Copy.7z was the batch file to translate the nc to grib(2) via CDO. IBI_ucurr.grib was the result (IBI_vcurr.grib was also produced but I find they have the same data).

Screenshots show the comparison:
Current.png is BSH
ibi_vcurr.png is Copernicus via CDO
ncf_plugin.png is Copernicus via netCDF.lib api plugin

All for the same time.

With the CDO grib using Grib2 plugin you see 2011-03-31 01:00:00UTC in the title bar. With ncdump the start date/time is found to be 2011-03-31 00:00:00UTC.

Again with CDO you see the earliest forecast time available is 1400. This corresponds to data for 1200 from both the ncdf plugin and BSH current data.

Writing the ncdf plugin I found that wxTimeSpan caused me a problem with the date 2011-03-31 00:00:00UTC. Adding the time span for the forecast produced a date/time one hour earlier than actual (as verified by a Web calculator). Worked around this by adding one hour to the forecast time spans (toUTC did not help). 31 March 2011 was Daylight Saving Time in Europe. Is this significant?

CDO.exe was compiled for Windows here, with grib_api support built in.
Perhaps someone on Linux can try the same exercise. Does did-g have an example.

Will the CDO route ever be used? Could this problem be ignored?

However the good quality data from Copernicus should not be ignored.

Mike
Attached Thumbnails
Click image for larger version

Name:	current.grb2.png
Views:	69
Size:	102.4 KB
ID:	120234   Click image for larger version

Name:	ibi_vcurr.png
Views:	54
Size:	150.6 KB
ID:	120236  

Click image for larger version

Name:	ncf_plugin.png
Views:	82
Size:	123.5 KB
ID:	120241  
Attached Files
File Type: pdf IBI_ucurr.grib.7z.pdf (563.4 KB, 11 views)
File Type: pdf copy.7z.pdf (267 Bytes, 12 views)
File Type: pdf Current_no_20160308.7z.pdf (267.1 KB, 14 views)
File Type: pdf ibi.nc.7z.pdf (479.4 KB, 10 views)
File Type: txt motu_client_script.txt (406 Bytes, 16 views)
__________________

__________________
Rasbats is online now   Reply With Quote
Reply

Tags
plug, rib

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
OpenCPN PlugIn Development bdbcat OpenCPN 78 29-07-2017 03:15
Plugin: DashBoard SethDart OpenCPN 470 24-08-2016 21:34
BSB 4 PlugIn bdbcat OpenCPN 93 20-02-2016 04:41
QTVLM Route Optimizer and GRIB2 sinbad7 OpenCPN 1 26-07-2013 01:13
Writing Plugin for nv.digital Map Format ? Netsurfer OpenCPN 7 17-08-2010 17:09



Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 01:12.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.