Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Rate Thread Display Modes
Old 30-08-2015, 17:37   #1
Registered User

Join Date: Jun 2015
Posts: 199
gribs concatenation and grib version2

Hi,

I pushed a POC https://github.com/did-g/OpenCPN.git
in branch WIP_next

- It uses a multiple files style dialog box, use CTRL and SHIFT for selecting files.

It does interpolation so you can open a 3h wind GRIB with 1h current GRIB and get some display (interpolated values are often wrong and it's slow but good enough for now).

If you open files with the same record type at the same time it will do, something..., surely stupid.

- I chainsawed stuff from qtvlm and ucar.edu for grib2, it's linux only for now.
You have to install jasper for jpeg decoding, on debian like system:
apt-get install libjasper1
apt-get install libjasper-dev

It's slow and uses a lot of memory but it's a start, no waves or precipitation yet.

Without jasper but with libgrib-api-tools installed you can convert from jpeg to
grid_simple encoding with:

grib_set -s dataRepresentationTemplateNumber=0 src.grb dst.grb

And still open grib2 files, as a matter of fact you can convert grib2 to grib1 with
grib_set -s edition=1 src.grb dst.grb
but it's buggy and there's not always a match between v2 records type and v1 type.

If gribs are too big you can split them with:
grib_copy src.grb dst[shortName].grb
and then open only dts<whatever>.grb
with whatever:
10u and 10v for wind (need both)
ucurr and vcurr for current
gust for, well gust (not ugust or vgust, ocpn doesn't use them)
tcc: total cloud cover.
msl: pressure

and so on.



- If you use Sean's weather_routing you have to clone:
https://github.com/did-g/weather_routing_pi.git

Internal grib record format is different and incompatible with older weather_routing versions.

The textbook Manche (English Channel) crossing COG example. model MeteoFrance AROME 0.025° cells and MyOcean current grib. (brown arrows are wind over water, black wind arrows are current).
Attached Thumbnails
Click image for larger version

Name:	Capture du 2015-08-31 01:00:52.png
Views:	96
Size:	92.9 KB
ID:	108152  
__________________

__________________
did-g is offline   Reply With Quote
Old 31-08-2015, 05:05   #2
Registered User
 
boat_alexandra's Avatar

Join Date: Aug 2009
Location: Masachusetts
Boat: bristol 27
Posts: 2,803
Re: gribs concatenation and grib version2

Quote:
Originally Posted by did-g View Post
Hi,

I pushed a POC https://github.com/did-g/OpenCPN.git
in branch WIP_next

- It uses a multiple files style dialog box, use CTRL and SHIFT for selecting files.

It does interpolation so you can open a 3h wind GRIB with 1h current GRIB and get some display (interpolated values are often wrong and it's slow but good enough for now).
Why is it slow, and more important, why is it "often wrong" ?
Quote:
If you open files with the same record type at the same time it will do, something..., surely stupid.
What to do in this case? Ask user which file to prefer, or?

Quote:
- I chainsawed stuff from qtvlm and ucar.edu for grib2, it's linux only for now.
You have to install jasper for jpeg decoding, on debian like system:
Is there a good reason we can't just start linking with libgrib and use that? Maybe it's too slow for us.
Quote:
- If you use Sean's weather_routing you have to clone:
https://github.com/did-g/weather_routing_pi.git

Internal grib record format is different and incompatible with older weather_routing versions.
I will look into this.
__________________

__________________
boat_alexandra is offline   Reply With Quote
Old 31-08-2015, 05:50   #3
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,369
Re: gribs concatenation and grib version2

Hey this is a start! Thanks for giving it a start. If we can just do a file concatination like file1 + file2 ^ then shouldn't it be pretty simple to do? Maybe libgrid is a good way to do it. Perhaps this should be part of the grib pluggin.
__________________
rgleason is offline   Reply With Quote
Old 31-08-2015, 05:51   #4
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,369
Re: gribs concatenation and grib version2

Thanks did-g for the effort.
__________________
rgleason is offline   Reply With Quote
Old 01-09-2015, 21:52   #5
Registered User

Join Date: Jun 2015
Posts: 199
Re: gribs concatenation and grib version2

Quote:
Originally Posted by boat_alexandra View Post
Why is it slow, and more important, why is it "often wrong" ?
It interpolates every missing records.
When interpolating, I don't use the right time for the new record...
I'm pretty sure it's also using the wrong math for current.

Quote:
Originally Posted by boat_alexandra View Post
What to do in this case? Ask user which file to prefer, or?
I don't know.

Quote:
Originally Posted by boat_alexandra View Post
Is there a good reason we can't just start linking with libgrib and use that? Maybe it's too slow for us.
by libgrib do you mean the one used in qtvlm?
Yes it could be used, it seems to do job, anyway it's a small change.
Speed shouldn't matter, most of the time is spent in jpeg decoding or bunzip2
__________________
did-g is offline   Reply With Quote
Old 01-09-2015, 22:31   #6
Registered User

Join Date: Apr 2015
Location: San Francisco
Posts: 65
Re: gribs concatenation and grib version2

I wrote some Python to generate curl to pull GRIB2 direct from NOMADS and found a tool from UCAR to convert them to GRIB1 for OpenCPN and/or zyGrib. Works fine, no chainsaw required.

Getting there required many days of looking at tools including Jasper, the Java tools, installing Fortran on my Mac, etc. But in the end it was pretty straightforward.
__________________
fogmachine is offline   Reply With Quote
Old 02-09-2015, 04:55   #7
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,369
Re: gribs concatenation and grib version2

Why can't grib_pi be improved to support concatinatipn and grib2?
__________________
rgleason is offline   Reply With Quote
Old 03-09-2015, 10:54   #8
bcn
Registered User

Join Date: May 2011
Location: underway whenever possible
Boat: Rangeboat 39
Posts: 2,758
Re: gribs concatenation and grib version2

Météo-France GRIB files finally for free – Navigation Mac

Another reason to get Grib 2.0 running....
__________________
bcn is online now   Reply With Quote
Old 04-09-2015, 00:46   #9
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,369
Re: gribs concatenation and grib version2

Looks good to me. Will be better and finer data for France and Europe than Noaa. Will provide cuonterpoint comparison to noaa for the world. Both models will get better.
__________________
rgleason is offline   Reply With Quote
Old 09-09-2015, 06:08   #10
Registered User

Join Date: Jun 2015
Posts: 199
Re: gribs concatenation and grib version2

Pushed a new version.

For testing you can download french meteo (AROME 0.025 cells, 36 hours) and IBI (myocean 0.033 cell) current at:

http://195.154.231.142/grib/arome
eg:
curl -o today.grb --compressed "http://195.154.231.142/grib/arome?x=-8&X=2&y=47&Y=51&r=cwgtprn"

with:
x: long min (neg for west)
X: long max
y: lat min
Y: lat max

and r records
c current
w wind 10m
g gust
t temp 2m
p pressure
r rain
n cloud cover
__________________

__________________
did-g is offline   Reply With Quote
Reply

Tags
rib

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Simple Iridium email and Weather gribs 9500 monte Marine Electronics 6 01-06-2014 21:25
Improving my gribs SailingMum Seamanship & Boat Handling 14 07-05-2014 17:06
gribs riddon OpenCPN 1 05-11-2013 07:40
Want to beta test ocean currents gribs? Eric. Navigation 5 29-01-2013 19:33
Grib Files and Grib Fantasies maxingout General Sailing Forum 4 16-01-2010 15:54



Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 00:20.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.