Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Search this Thread Rate Thread Display Modes
Old 24-05-2017, 06:39   #16
Senior Cruiser

Cruisers Forum Supporter

Join Date: Aug 2009
Posts: 4,037
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

^^ you do know that you can get multi-day rtofs file downloads by using Jim Corenman's very easy to use graphic interface called ViewFax (despite the name it does grib downloads not faxes)? You dont need any scripts. It comes bundled in with the airmail program if you already have that, or you can download seperately (get the beta version) free.
__________________

estarzinger is offline   Reply With Quote
Old 24-05-2017, 12:20   #17
Registered User

Join Date: Oct 2014
Posts: 203
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

Quote:
Originally Posted by rgleason View Post
Actually I think XXX defines the max hours wanted so the file name would be
rtofs_glo_2ds_f144_3hrly_prog.nc

Just need surface currents and a lat/long box next.

Now going to
ftp://ftpprd.ncep.noaa.gov/pub/data/...tofs.20170524/

I see that these files are huge. More later.
Rick,

I think the "rtofs_glo_2ds_fxxx_3hrly_prog.nc" files have the surface current data you want, but that FTP server (in the quote box above) only has these files with forecasts out to 96 hours (rtofs_glo_2ds_f096_3hrly_prog.nc). If you use the NOMADS page to access the "RTOFS Global" data by clicking on the "HTTP" box, you can get the forecasts out to 192 hours (rtofs_glo_2ds_f192_3hrly_prog.nc).

If you click on the "HTTP" column heading (at the top of the matrix) you are taken to instructions for getting partial downloads of the GRIB data, but there are no instructions for downloading a regional subset (latitude and longitude limits). I think you found this, but I mention it to help someone else. I have not tried this.

If you click on the "gds-alt" column heading (top of matrix) you get instructions for downloading by data name and for a region bounded by latitudes and longitudes. If you go this route, you access the files by clicking on the "OpenDAP-alt" box to the right of "RTOFS Global". I have not tried this.

Paul
__________________

.Paul. is offline   Reply With Quote
Old 26-05-2017, 06:53   #18
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

Evans, Thank you. I did not know this. Earlier I was going to try that and got sidetracked.
Quote:
you do know that you can get multi-day rtofs file downloads by using Jim Corenman's very easy to use graphic interface called ViewFax (despite the name it does grib downloads not faxes)? You dont need any scripts. It comes bundled in with the airmail program if you already have that, or you can download seperately (get the beta version) free
Paul.
I will give that location a try. Also perhaps use of Viewfax will shed some light on the sources.

I've also been looking into PredictWind 3month Standard and Professional for the purpose of receiving 3hr wind and current data via Iridium Go, and it appears it would work fine and give the Skipper's Mac a routing using one polar file which would be great, but they do not yet cover the Gulf of Maine for "currrents". Tech advises it will be 1-2 months. Tech Support has been very helpful though.

Any suggestions are welcome.

I note here again that:
1. Opencpn Grib_pi Downloads using zygrib username and password is no longer available, so this download feature appears to be broken and should be fixed.
2. Opencpn Grib_pi Downloads using Saildocs with Rtofs request results in 24 hr gribs even when 12 hr is requested. Additionally 3hr Rrtof current gribs are available so that should work too, but Saildocs grib server has been affected by the changes at NOAA to using the RTOFS Global Model, thus breaking the Saildocs Rtofs download.
3. Since Grib_pi has been updated to grib2 and since we can load two grib files now, it would be truely wonderful to update this handy and essential plugin for improved downloading.
4. I have found the "walking or moving grib" feature invaluable for reducing the size data downloads and hope that at some point download of Rtofs currents would work in the same way with 3hrr resolution.

-PS I will post on Flyspray and Github.

On to try ViewFax!
rgleason is online now   Reply With Quote
Old 26-05-2017, 08:34   #19
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
ViewFax - Saildocs- download of Rtofs --Noaa Server has changed.

Dear Evans,

Things have changed. I've tested Viewfax and it is downloading only 24hr RTOFS data now, even though the interface allows 12hr selection.

This is the same situation with the OpenCPN Grib_pi Download because Grib_pi is also using Saildocs. The change that has caused this is NOAA's change in servers.

In short Saildocs needs to revise the RTOFS download source to the new servers, and while doing so, perhaps they can provide 3hr interval because these are available now.

So the fix is becoming clearer now. Can someone advise Saildocs? Or is there someone who is involved who can get this change made?
rgleason is online now   Reply With Quote
Old 26-05-2017, 09:45   #20
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

This is a bit like groping around in the dark, but Paul's and Evans help is much appreciated.

I think I found the necessary data on the NOAA Global RTOFS Product>Data Access server:
ftp://ftpprd.ncep.noaa.gov/pub/data/...tofs.20170525/

down towards the bottom:

rtofs_glo_2ds_f192_3hrly_prog.nc
405842 KB 5/25/17 4:04:00 PM
This file needs to have a lat/long area and only surface current data extracted. There must be a method to do this.
---

Using Paul's suggestions:
HTTP

1. Use Nomads: NOMADS-NOAA Operational Model Archive and Distribution System
2. Click on "http" leads to instructions for partial downloads.
"Using this procedure, you can download just the data you need from a remote GRIB2 file." Fast Downloading of GRIB2 Files The "Details" paragraph indicates that several linux programs are needed. "The wgrib inventory is used for the index file, and cURL is used for the random-access http program. " and two perl scripts "Two perl scripts are also required, get_inf.pl and get_grib.pl, which are downloadable from the NCEP CPC website (see "Requirements" below)." There are some examples as well. It appears this would be a linux server function.
  1. perl
  2. grep
  3. cURL
  4. grib files and their wgrib inventory on an http server
  5. get_inv.pl
  6. get_grib.pl

There are several examples of Windows users who have configured Perl, curl and wgrib so they can download partial files. I don't know if I will try this yet I am using Perl scripts now.

At the bottom there is a note about Regional Subsetting
"Right now, I am happy with the g2subset software that is running on the nomads servers. "

The other alternative GDS-ALT

1. Use Nomads: NOMADS-NOAA Operational Model Archive and Distribution System
2. Paul suggests click on the "gds-alt" column heading (top of matrix) you get instructions for downloading by data name and for a region bounded by latitudes and longitudes.

I think I'll try this first. __I guess not. Configuring a Grads Data Server -DODS is well beyond my capability.

DODS
"
The GrADS-DODS server is a stable, secure, easy to install, and easy to configure data server that provides subsetting and analysis services across a network.
The subsetting capability allows a user to request a specified range of time and space from a large dataset, eliminating the need to download everything simply to access a small relevant portion of a dataset."

I think this is the GrADS data format needed...
http://nomads.ncep.noaa.gov:9090/dod...aily_prog.info

u_velocity: Grid_FillValue: 1.26765E30
missing_value: 1.26765E30
long_name: "eastward_sea_water_velocity (m/s) "

v_velocity: Grid_FillValue: 1.26765E30
missing_value: 1.26765E30
long_name: "northward_sea_water_velocity (m/s) "

But I don't see where the data is located and am not connecting the dots....
rgleason is online now   Reply With Quote
Old 26-05-2017, 10:15   #21
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

I have to say, that I've reviewed this for windows and looked at wgrib and the notes, and I could try to set this up sometime but there must be an easier way!

I wish svsarana website were reconfigured for the new NOAA setup. That would be very useful, but that website also does only 24hr RTOFS current now too.
rgleason is online now   Reply With Quote
Old 26-05-2017, 12:45   #22
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

Check out https://www.gribfiles.com/#/Weather-models
RTOFS TAB They are using HYCOM & offer what we need:

RTOFS Type of model Regional Ocean Forecast System, HYCOM Geographic area Inside the red box at the frontpage map Model grid (horizontal, km)
Grib file grid (horizontal, deg) 0,083 x 0,083 Timespan (hours) 144 Time step (hours) 3 Available parameters Surface current
Initialization time (UTC) 00:00 Available from server (app, UTC) 17:00 Source of data NCEP - NOAA Remarks
Use for Give a brief overview of the ocean system
Do not use for Tidal streams and ocean currents for coastal areas Comments
rgleason is online now   Reply With Quote
Old 26-05-2017, 13:29   #23
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

NOAA notice: Rtofs Atlantic ended, superceded by Global Real Time Forecasting RTOFS Global.

Quote:
NOTICE: Effective on or about March 14, 2017, the National Centers for Environmental Prediction (NCEP) will discontinue the North Atlantic Basin Real Time Ocean Forecasting System (RTOFS Atlantic). This system has been superseded by the Global Real Time Ocean Forecasting System (RTOFS Global), which became operational October 2011. More information is available here.
rgleason is online now   Reply With Quote
Old 26-05-2017, 14:35   #24
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

I should have gone to the notice of termination earlier.
http://www.nws.noaa.gov/os/notificat...fs_removal.htm
which suggests this page for replacement of Atlantic Rtofs
WAVE Products

"RTOFS HYCOM - Global 2 Dimensional Surface Prognostic- 1/12 Degree Resolution Forecast - 3 hrly
rtofs_glo_2ds_fxxx_3hrly_prog.nc FH00-192

For example, todays version found in this directory:
http://www.ftp.ncep.noaa.gov/data/nc...tofs.20170525/

Now how to extract the lat-lon area and just the surface currents?
rgleason is online now   Reply With Quote
Old 26-05-2017, 14:59   #25
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

Now that I've downloaded 410mb, I see that this file is a netcd file which ends with .nc and it is probably not compatible with Grib_pi.

However Mike Rossiter was working on a Netcdf current reader earlier, maybe that will work, but it won't work with WRouting....
NCDF_pi works on .nc files but chokes on this 410mb file.

This isn't going to solve my problem.
rgleason is online now   Reply With Quote
Old 26-05-2017, 19:33   #26
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

Has anyone tried partial HTTP downloads from NECP Nomads with an OS = Windows using:

Requirements
  1. get_gfs.pl.
  2. perl
  3. cURL
https://curl.haxx.se/download.html

Refer to

Fast Downloading of GRIB2 Files
Fast Downloading of GRIB Files, Part 2

NOMADS-NOAA Operational Model Archive and Distribution System
These operations would be run on files similar to:
Index of /pub/data/nccf/com/rtofs/prod/rtofs.20170525
rgleason is online now   Reply With Quote
Old 29-05-2017, 14:12   #27
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
Global RTOFS - need for Grib Filter System!!!

In a nutshell, here is what happened at Noaa.
Atlantic Real-Time Ocean Forecast System (Discontinued) Notice effectiv March 14.

Follow the link to Global RTOFS Notification http://www.nws.noaa.gov/os/notificat...fs_removal.htm

More details about the suite of products from the Global RTOFS
that replaced the legacy RTOFS Atlantic are available at
WAVE Products

See
rtofs_glo_2ds_nxxx_3hrly_prog.nc

See this note:
The Inventory links contain detailed model information
Use as a guide when selecting specific parameters from NOMADS through the grib filter option

Which leads to the NOMADS Grib Filter page
NOMADS-NOAA Operational Model Archive and Distribution System

Down the list under "Ocean Models"

RTOFS Atlantic daily grib filter http OpenDAP-alt

RTOFS Atlantic HIRES daily grib filter http OpenDAP-alt

RTOFS Global
daily - http OpenDAP-alt

Rtofs Global has no "Grib Filter" and the RTOFS Atlantic are now cancelled.
I hope this changes soon!

Using the Grib Filter this is a typical URL=
URL=
http://nomads.ncep.noaa.gov/cgi-bin/...2Fofs.20170321

I wonder if it can be used & modified for the new global rtofs?

GrADS Data Server - info for /rtofs/rtofs_global20170528/rtofs_glo_2ds_forecast_3hrly_prog

and then adding something like:
&lev_0_m_below_sea_level=on&var_UOGRD=on&var_VOGRD =on&subregion=&leftlon=-71&rightlon=-68&toplat=44&bottomlat=41&dir=%2Fofs.20170321
from the Grib Filter above? -- but I don't think this is the correct server and I haven't got the correct filter defined..

Perhaps we could write noaa.. the emails are right there.

Alternatively, does anyone know how to extract data from a grads server?

rgleason is online now   Reply With Quote
Old 29-05-2017, 20:41   #28
Moo
Registered User

Join Date: Mar 2017
Posts: 338
Re: ViewFax - Saildocs- download of Rtofs --Noaa Server has changed.

Quote:
Originally Posted by rgleason View Post
So the fix is becoming clearer now. Can someone advise Saildocs? Or is there someone who is involved who can get this change made?
Can't you advise Jim Corenman of your findings? His contact is in Airmail. He is very helpful.

Cheers
Moo is offline   Reply With Quote
Old 30-05-2017, 06:12   #29
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 12,555
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

Thanks, Have sent an email to airmail3 #at airmail2000 #dot com
rgleason is online now   Reply With Quote
Old 30-05-2017, 14:07   #30
Moo
Registered User

Join Date: Mar 2017
Posts: 338
Re: Use of RTOFS for Current.Gribs - Gulf of Maine

Quote:
Originally Posted by rgleason View Post
Thanks, Have sent an email to airmail3 #at airmail2000 #dot com
Have sent you a PM Rick.
__________________

Moo is offline   Reply With Quote
Reply

Tags
current, maine, rib

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
St Lawrence Seaway, Gulf of St Lawrence, Nova Scotia, Gulf of Maine sailing time wind Jm2fly General Sailing Forum 9 11-07-2016 21:53
Avoiding Gulf Stream Current Headed S pete33458 Atlantic & the Caribbean 56 07-09-2015 16:30
RTOFS Gulf Stream Model, Saildocs and Viewfax msulc Navigation 2 08-09-2011 20:08
Gulf Stream Counter Current / Northwest Cuba ? alaskadog Atlantic & the Caribbean 2 22-08-2005 16:51



Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 19:52.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2019, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.