Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Rate Thread Display Modes
Old 25-01-2013, 17:37   #1
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Pilot Grib Files - Similar to Pilot Charts

I had a question from a friend about planning some long trips with grib files. He has become familiar with OpenCpn and qtVlm. He asks if it is possible to use qtVlm to simulate routes during different times of year?

1. Pilot charts perhaps which OpenCpn has available, but not in a grib format?
2. Use a public archive of Grib files to create "recorded" grib data for different times of the year (typical weather at those times of year).. These would be looking back, not forward, not "predictive" but "recorded". - Pilot Gribs (a full year of the entire world, typical "recorded" weather).

Pilot charts show the average wind speed & direction, whereas these Gribs just show what happened. At some point there could be an averaging of some sort.

Then Maitai had a great suggestion on La Taverne, sending me to a link to a website:
Routing in the past with Old Gribs.

This appears to be a piece of software that appears to compile an entire day from the most accurate hourly gribs for each hour, so that an optimal routing can be done using gribs from dates/times in the past. Despite the clever software, it is going to be a fair amount of work to compile this data.

I am wondering if anyone knows of such a Recorded Routing Grib repository which has this already compiled by year for the world?
OR perhaps there is an easier way to get this complied for last year?

Thanks
Rick

Possible OpenCpn Feature sometime in the future?:
-Display of weather fax (qtVlm does this) and satellite weather in layers with the gribs.
- Join two Saildocs downloads together to reduce the volumn of data... or perhaps to be able to define a polygon with multipple points request to Saildocs. May be more dependent on Saildocs.

__________________

__________________
rgleason is offline   Reply With Quote
Old 26-01-2013, 01:06   #2
Senior Cruiser
 
sailorF54's Avatar

Cruisers Forum Supporter

Join Date: Dec 2009
Location: Perros-Guirec, France
Boat: Jeanneau Sunshine 36
Posts: 829
Re: Pilot Grib files - similar to Pilot Charts

IMHO, getting the raw data from the CoGow project (end results already available openly here
Climatology of Global Ocean Winds - COGOW) and putting it in grib format would be much simpler and comprehensive
Attached Thumbnails
Click image for larger version

Name:	Image005.jpg
Views:	244
Size:	264.5 KB
ID:	53498  
__________________

__________________
sailorF54 is offline   Reply With Quote
Old 26-01-2013, 02:32   #3
Registered User

Join Date: Dec 2005
Location: WNA
Boat: Dufour 35
Posts: 3,247
Re: Pilot Grib files - similar to Pilot Charts

Cogow is the way to go, for now. Why bother squeezing more data or formats from something that haven't been updated for more than 40 years?

Cogow can bee displayed on OpenCPN, in a fashion, already.
Check out this guy Prevailing Ocean Winds Pitufa.at. Cogow in kml format! Not perfect, but interesting.
Click image for larger version

Name:	cogow_kml.png
Views:	449
Size:	233.9 KB
ID:	53499

When it comes to displaying wx faxes, OpenCPN has done that for years. All wx fax maps in Mercator, including all US faxes, are taken care of by a script and converted to kaps. Check out old thread(s) about this. "Nohal" used to run a script on his server, creating fresh kaps every day for the Atlantic and the Adriatic. I can't find them now...! A sample

Click image for larger version

Name:	faxkap.jpg
Views:	833
Size:	439.0 KB
ID:	53500

The idea of aggregating old gribs seems flawed to me. All the weaknesses and errors in gribs would still be there. To be of any value, updated or modern pilot charts, must continue to be based on real weather observations.

Thomas
__________________
cagney is offline   Reply With Quote
Old 26-01-2013, 08:26   #4
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

Thanks for the suggestion about Weatherfax overlay in Opencpn. Also COGOW which may be just what is needed.

Thomas, just a reminder, the title is "Pilot Grib" not based on "predicted" but on "actual" historical past - lets say real data instead. That was the whole point...use the gribs that are closest to what actually happened...everything is an approximation anyway, real weather observations are an approximation too.

I agree there are problems with "aggregating" and "averaging" gribs. How do you do it effectively? Perhaps the Cogow is the best thing to try to use for routing. Perhaps compile that information in grib format somehow for the time period of the trip? - I am going to have to read about this more.

I was just trying to get some past grib files (closest to real weather data as possible) that would allow me to run some routes in qtVlm at certain times of year, to answer my friend's question. Rather than complie (and aggregate) the most accurate gribs from last year, perhaps I should look into using Cogow for this.

Thanks both of you for the perspective and suggestions.
The downloads at http://www.pitufa.at/2010/12/prevailing-ocean-winds/ in kml format look good too.
__________________
rgleason is offline   Reply With Quote
Old 26-01-2013, 09:09   #5
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

From JeanLouis on La Tavern. Thanks JeanLouis.

Hi, Rick !
Your 2 links are wrong....
Routing in the past:
http://wiki.virtual-...iens_(archives)

Tool cygwin:
http://www.iges.org/.../downloads.html
__________________
rgleason is offline   Reply With Quote
Old 26-01-2013, 09:17   #6
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

Cagney writes:
Check out this guy Prevailing Ocean Winds Pitufa.at. Cogow in kml format! Not perfect, but interesting.
------
The author writes:
...have generated kml files for the Atlantic, the Indian, and the Pacific Ocean, one file per month of a year. The files contain prevailing wind arrows at a 2-by-2 degree lon/lat grid. The direction and length of an arrow was obtained by vectorially averaging monthly data over 10 years and over 2-by-2 degree areas (i.e., 32 COGOW data files were averaged for one arrow). The color of an arrow corresponds to the scalar averaged wind speed in knots. In addition, each arrow links to a wind rose image located at COGOW.
----

Is there any way to import this KML into gtVlm which turning it into a longer term grib? Say one for each month?
So we could use it for qtVlm Pilot Grib routing purposes? Testing different routes based on averages?
__________________
rgleason is offline   Reply With Quote
Old 26-01-2013, 10:42   #7
Registered User
 
Viking Sailor's Avatar

Join Date: Nov 2006
Location: San Francisco Bay
Boat: Fantasia 35 - s/v Feeling Good
Posts: 1,074
Re: Pilot Grib files - similar to Pilot Charts

At COGOW (Prevailing Ocean Winds Pitufa.at.) Christian writes:

Quote:
Download this kmz file (= zipped kml files) to get a world-wide prevailing sea wind atlas which is convenient for off-line usage. You may simply load it into your Google Earth (File > Open…). In your ‘My Places’ panel you will get a list to switch between the months. Displaying them right after loading may take some time. Repeated displaying should work fine.
Might it be possible to use GE2KAP to convert and overlay the GE images as KAP charts in OpenCPN?

__________________
Viking Sailor is offline   Reply With Quote
Old 27-01-2013, 11:34   #8
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

(Prevailing Ocean Winds Pitufa.at.) At Map Pitufa.at Select Winds in the upper right then Check North America, and a Month.

This looks very much like it could be used for Pilot - Grib Routing. Is it possible to import as a grib or to convert to grib somehow? The ability to click on a location to see the Wind rose and winds predicted is nice but not essential. This is basically a grib file for the month! Pretty simple.
Attached Thumbnails
Click image for larger version

Name:	Pitufa.at-with-Wind-selected-COGOW.png
Views:	241
Size:	325.7 KB
ID:	53560  
__________________
rgleason is offline   Reply With Quote
Old 27-01-2013, 17:42   #9
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

These montly files at Map Pitufa.at , say "prev_winds_north_atlantic_01.kml" are Xml format with the main repetitive part being "Placemarks"

<Placemark>
<name>8.47 kn scalar average</name>
<styleUrl>#tip2</styleUrl>
<LineString>
<coordinates>
8.273842, 1.166456
8.336253, 1.408540
8.110688, 1.300741
8.273842, 1.166456
</coordinates>
</LineString>
<description>
<![CDATA[
<a href="http://cioss.coas.oregonstate.edu/cogow/0101/gif/1.25N_8.25E.gif">View wind rose.</a>
]]>
</description>
</Placemark>


The actual arrows that make up the file are referenced this way
<styleUrl>#tip2</styleUrl>
with each type of arrow being described more completely at the top of the file.

The <description> CDATA link to wind roses would not be needed if it complicated things.

This is a pretty simple structure, with several Lat, Long and references. It may be formatted in accord with OpenKML format but I don't really know.

There must be a pretty direct way to translate to grib format. Should it be Grib1 or Grib2 format?
__________________
rgleason is offline   Reply With Quote
Old 27-01-2013, 18:03   #10
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

Under Pilot Charts this website has some great big PDF Pilot Charts that might be of interest to someone, but not us Sailing Weather - Marine Weather Forecasts for Sailors and Adventurers - PassageWeather . I'd like to get some Pilot Grib files that can be read by qtVlm and OpenCpn.

There is evidently a process to create KML files like above from Grib files. We just need to reverse that.
__________________
rgleason is offline   Reply With Quote
Old 27-01-2013, 18:26   #11
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

Tools and possible Source data and process.
Grib <-- KML or
KML --> Grib or
NOAA Products --> Grib (Ie take Noaa's best grib's from the past, say the 1 hour forecast and string them together for full months. This would be an example and not a Pilot Grib.

1. Take raw data from the CoGow project (end results already available openly here Climatology of Global Ocean Winds - COGOW) and putting it in grib format would be much simpler. -sailorF54
2. Prevailing Ocean Winds Pitufa.at. Post processed and averaged Cogow in kml format! --Not perfect, but interesting. Cagney.

Possible Tools
PyAOS - Reading Grib on Windows
NOAA's Weather and Climate Toolkit
CygWinWindow


This is getting a little deep I am afraid. It certainly is not importing a KML Route into OpenCPN, but it seems with some database transformation tool we could easily get these into Grib format that could be read by OpenCpn and QtVlm.
__________________
rgleason is offline   Reply With Quote
Old 27-01-2013, 18:40   #12
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

Weather4D - How do these guys do Historical Grib Files (under features), rather than forecast.
KML Output Output (Macro-python)
ECMWF Grib API - this is a little beyond me.

Perhaps if I knew what a Grib format was (it looks binary!!) I could use Notepad++ to work the KML file into the right format?
__________________
rgleason is offline   Reply With Quote
Old 27-01-2013, 18:51   #13
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

Dave about possible use of KML in OpenCpn, (KML Route is done).

Quote:
I plan to add generic KML overlay capabilities to opencpn. Reading and parsing XML is straightforward with wxWidgets, so I have a starting hook.

Initial plan of attack and proof of concept(simplified): Export a track from opencpn as GPX. Convert to KML. Validate on GE. Write code. Import this track KML to opencpn as overlay. Compare to original track. If all goes well, then...

The next use I have in mind is to generate intermediate KML files from GRIB data as part of the GRIB effort. These intermediate files will then be loaded into opencpn as semi-transparent overlays.

Of course, as a side benefit, these GRIB->KML files could be read in Google Earth, as well. Seems like a useful thing. Maybe I'm not the only one working on this particular path... ...he then mentioned something to the effect that this plan is like jello....
.
__________________
rgleason is offline   Reply With Quote
Old 27-01-2013, 19:15   #14
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

Create a long term 'historic' Actual weather Grib
OBJECT: Collect successive forecasts due immediately so as to constitute a grib file reports, not a long-term forecast, but the weather actually did the sea over a long period.
1. VLM Servers have GFS weather model data stored.
2. ConvGrib Utility - (NOAA) Convert Grib -->Grib2 OR Grib2 -->Grib
3. First Goal Reconstruct a full day with the best actual gribs.
- Select the best gribs and short prediction times.
- Concatenate each of the gribs into one file
4, Procedure is outlined in link very well.
5. A. Grib day based on 8 files. GRIB2 weighs about 3 MB

It would take some time to get a month together and it would be quite big.

It might be simpler and perhaps more useful to take the KML files from
COGOW (Prevailing Ocean Winds Pitufa.at.) and convert them for each month.
The file would be smaller, and routing could be done with some "Fat" route to represent the degree off variance.

A further development might have an approximation based on some number 1-10 for the wind variability in speed and direction at that time of year at that point.
__________________
rgleason is offline   Reply With Quote
Old 02-02-2013, 20:49   #15
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,305
Re: Pilot Grib files - similar to Pilot Charts

Sailover in La Taverne thread below, post 7260
qtVlm - Page 363 - Outils de navigation pour VLM - Navigation tools - La Taverne

Suggests:
Quote:
A possible way could be the using of the "Wgrib2" (1) utility with the "-import_text data.txt" (2) and the "-grib_out file_name" (3) commands

(1) http://www.cpc.ncep..../wesley/wgrib2/
(2) http://www.cpc.ncep....mport_text.html
(3) http://www.cpc.ncep....2/grib_out.html


I never experienced this way but you should obtain a grib2 file from a KML one throught the following steps:
- Convert KML file into a text file (using KMLCSV utility),
- Convert text file into Grib2 file.
To know what is the requested text file data arrangement, you could convert a grib2 file into a text file using the WGrib2 utility.
Looks like it might be possible to convert KML. Has any one here experience with this? Is it difficult? Thanks.
__________________

__________________
rgleason is offline   Reply With Quote
Reply

Tags
charts

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Charts II: NGA - 2700 Charts cagney OpenCPN 1292 23-12-2013 09:46
CM93 Charts Up to Date ? tonydix Navigation 7 29-11-2011 12:53
DNC / VPF Format Digital Nautical Charts Kudalufi OpenCPN 0 14-08-2011 15:01
Route Properties, Missing Functions James Baines OpenCPN 13 13-07-2011 05:31
Want To Buy: Wanted: Charts, Pilot Guides for East Coast US and Canada annk Classifieds Archive 0 05-07-2011 09:00



Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 18:19.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.