Climatology - Database Search

[rgleason]

Sean has expressed an interest in adding several types of data to the Climatology Database. If you have a very good data source for any of these data types, please add a post listing the Type of data, URL link, and why you think it could be used. We are looking for these data types specifically, as we do not want to increase size too much, but good suggestions may be considered. Please post the DB info to this post, not the other Climatology post!

Fog: Montly % chance of heavy (<1nm), medium(>1nm<2nm) or light fog, probably from historical records, not calculated because it will not be accurate enough. - wrote to a weather scientist. Experimental

Maritime Distress Alerts (non-false): Average density, by month, maybe several types of distress.

• COPS - SARSAT - Mcc's have databases.
• Cops-Sarsat List of Alerts -good
  
• GMDSS Global Maritime Distress and Safety System
• EPIRB, SART, DSC
• Inmarsat - written to - Brian Mullen "Inmarsat opens up its safety database"

Heavy Ship Traffic Areas: Perhaps from some historic AIS database. How to measure density of traffic? by Tons of shipping? Like marinetraffic.com but historic.

Historic Trade winds Routes and Jimmy Cornell Routes: Clipper routes, Historic Sailing Trade routes, Jimmy Cornell Routes need to be compiled.

[rgleason]

ATOVS, Aerosol, Cloud and Radiation products. Appears to be stored daily, may be similar to current Clouds data.

Experimental GOES Fog image -Not a database yet, but the sensors are there!
Regional GOES Fog image - for US, we need worldwide database, for a year, 10yrs etc.

STAR / SMCD / OPDB
STAR / SMCD / OPDB - ATOVS Aerosol, Cloud & Radiation Products
http://www.star.nesdis.noaa.gov/smcd...y_coverage.php

[rgleason]

Copy of some email discussion with a Noaa Scientist about developing a world monthly probability of fog for the Climatology database. It looks favorable and possible.

Rick,

Thank you for further explain your plan. I agree there are issues that affect your fog climatology computation.
I am really interested in your gridded fog climatology data. If it is successfully generated it'll be very useful for
us and coast weather forecasters as well. For us it could be used to generate "re-forecast" here at NCEP.
For many other weather, "re-forecast" is conducted based on their climatology data. But for fog or visibility we
have no such data yet.

For monthly, climatology fog data coarse grid data should be OK since you need not to know detailed fog physics inside
each event in climatology statistics computation. For dew point computation, it is sometimes tricky because of different
formulations in saturation vapour pressure calculation. Using monthly air temperature, sea temperature, RH and wind
to derive likelihood of low visibility and fog is, I believe, the most practical method if the derived likelihood levels (percentages)
can be adjusted by some observational data. Using climatology data to predict specific fog events is an issue since significant
info inside the data have been smoothed. This does not mean there is no chance but need to verify.
Another issue is that reduction in surface visibility is not only by fog, it is also by other weathers like precipitation,
drizzle, snow, etc. Do your database also consider these events?

Binbin



On 05/29/2013 10:04 PM, Rick Gleason wrote:

Dear Binbin
Thank you for your thoughtful and informative response. The data sources so far are listed below."Our fog prediction from NCEP/NWS/NOAA is basically from diagnosis of operational model outputs using following variables:
(1) visibility or (2) surface wind, cloud base/top, plus RH using some thresholds. Are your prediction data also obtained from
some NWP (Numerical Weather Prediction) models? - Yes Since different models have different system bias. Using dew point
should be fine to replace RH but its threshold may be varies from model to model. - Ok
Your idea of generating a monthly fog climatology database is also encouraging and will be very useful. Although there
will deviation (anomly) from the average over a region, fog occurrence probability can be inferred from this monthlymean value. What is the horizontal resolution for this monthly mean value data? -3 degrees I believe, Sean will know. Could it be put into a gridded data
format like GRIB1 or GRIB2?" -Yes it is in gridded data

Fog prediction from a Multimodel Mesoscale Ensemble Prediction System
http://www.emc.ncep.noaa.gov/mmb/SRE...0.official.PDF
"For operational models such as at NCEP, model resolutions are still too coarse to properly resolve important physics for fog near the surface due to computing resource limitations."
-In our case, developing monthly climatological data for low fog probability (dense fog that affects visibility (1 nautical mile or less, 2 nautical miles or less) would we have to use such computing expensive small grids? Because the calculations will probably have to be done on my 1yr laptop or my older desktop. Additionally we can download data and have taken time to download approx 170gb for the data below, but there are some practical limits.

What formula would you suggest for calculating dewpoint? Hopefully using average monthly Air temp, Sea temp and Relative Humidity and possibly some others like wind? If dewpoint is hit using the monthly averages, is that reasonable to infer that there will be a high likelihood of fog and low visibility? -We can also assume there will be times and locations where the conditions are extreme and do not conform to averages, where there will be fog. Indeed sometimes the worst fog conditions occur during these times, and average data will not capture that variability, or the proclivity towards that kind of variability. Are there some simple ways to use the average data plus some measurement of the frequency (% time value is at some high or low range) and variability (high and low range for that month) to capture those fog events, in a relative and imprecise, but useful way? The goal being to establish %chance of occurrence of fog and low visibility? You know perhaps you have a visibility database of the world. I note item (1) Visibility --That would be the key that we are trying to get in the database!! Maybe Noaa has already done this number crunching and mathmatics.

I mention that there is bouy data available which might be a way to calibrate.

Your help thoughts and assistance would be most welcome, and we appreciate your response to our first note!

Thank you, Rick Gleason




Sources of Climatology Data

Ocean Depth ?
Humidity ?
Precipatation ?
Fog ? Seatemp+Airtemp+Humidity
Wave Height ?
Swell ?

Tropical Cyclones - Best Track Database
IBTrACS - International Best Track Archive for Climate Stewardship

Pressure
slpcoadsclim5079.nc (Make -k downloads it)
http://www.jisao.washington.edu/data...adsclim5079.nc

Sea Temperature
sstcoadsclim6079.1deg.nc (Make -k downloads it)
http://www.jisao.washington.edu/data...im6079.1deg.nc

Clouds
cldccoadsclim5079.nc (Make -k downloads it)
http://www.jisao.washington.edu/data...adsclim5079.nc

Currents - Small files
C:\Data-Dart\Climate-Current-Small
102 files about 9449kb each, running
From: oscar_vel7001.nc.gz to oscar_vel7514.nc.gz
ftp://podaac-ftp.jpl.nasa.gov/allDat..._vel7001.nc.gz and down.
Note: These files must be extracted. GZ files cannot be read by Gendata/Makefile!! (make -k)

Currents - 2009,2010,2011,2012,2013 yearly files
C:\Data-Dart\Climate-Current
2009,2010,2011,2012,2013 in single large files
From: ftp://podaac-ftp.jpl.nasa.gov/allDat..._vel2009.nc.gz
To: ftp://podaac-ftp.jpl.nasa.gov/allDat..._vel2013.nc.gz

Wind Daily 1980s - 2000s
C:\Data-Dart\Climate-winds-Daily
ftp://eclipse.ncdc.noaa.gov/pub/seaw...uv/daily/ieee/

Winds 6hr 1980s-2000s
C:\Data-Dart\Cllimate-winds-6hr
ftp://eclipse.ncdc.noaa.gov/pub/seaw...ly/ieee/2000s/
ftp://eclipse.ncdc.noaa.gov/pub/seaw...ly/ieee/1990s/
ftp://eclipse.ncdc.noaa.gov/pub/seaw...ly/ieee/1980s/



On 5/28/2013 10:41 AM, Binbin Zhou wrote:

Rick,

Thank you very much for let's know your "Opencpn Navigation Program". In my thought your idea for prediction of fog
is generally right. My background is fog prediction on land, but sea fog or coast fog and fogs on land have similar features.

Our fog prediction from NCEP/NWS/NOAA is basically from diagnosis of operational model outputs using following variables:
(1) visibility or (2) surface wind, cloud base/top, plus RH using some thresholds. Are your prediction data also obtained from
some NWP (Numerical Weather Prediction) models? Since different models have different system bias. Using dew point
should be fine to replace RH but its threshold may be varies from model to model.
Your idea of generating a monthly fog climatology database is also encouraging and will be very useful. Although there
will deviation (anomly) from the average over a region, fog occurrence probability can be inferred from this monthly
mean value. What is the horizontal resolution for this monthly mean value data? Could it be put into a gridded data
format like GRIB1 or GRIB2?

The prediction of fog from NCEP/NOAA is still not so successful because of complexity of different types of fogs.
NESDIS (of NOAA) has some products to detect low cloud stratus over North America but right now only images can be
available to us, no digital data files yet.

Due to uncertainties in NWP particularly in the initial conditions and fog physics, ensemble forecast tech has been tested
in fog prediction. If you have enough resources, you also can try this kind of technique. For this application we have another
paper "Fog prediction from a Multimodel Mesoscale Ensemble Prediction System" on Weather and Forecasting, 303-322, 2010.
If you are interested in ensemble, I can show you where NCEP's ensemble forecast data are located so you can download
them (both global and regional, free, open to public). In general, ensemble mean fields (temperature, humidity, wind speeds,
etc) are more accurate than single models.

Last, how to calibrate and evaluate your fog detection method is another concern. On land this can be done with surface
station data. But over seas, you have to have many sea fog cases to compared with. If you have enough observations on
ships, this won't be a question for this.

Binbin Zhou
Environment Model Center (EMC) of
National Centers for Environmental Predition (NCEP)
National Weather Service, NOAA




On 05/26/2013 11:14 AM, Rick Gleason wrote:

Dear Binbin,

We are developing an open software plugin for Opencpn Navigation Program that we call Climatology, which allows Routing predictions for sailors based upon average data for wind, currents and various other factors. Many of these functions now work. See
http://www.cruisersforum.com/forums/...ml#post1243712
http://www.cruisersforum.com/forums/...ml#post1242617

The routing is based upon compression of some worldwide databases across many years for wind, currents, airtemp, seasurfacetemp, humidity, airpressure, waveheight, etc. We took about 180gb of data and compressed it down to about 8mb to get monthly averages. We are considering adding some data for variability of wind and more % of time at a certain direction which will increase size somewhat.

We are now looking for Climatology databases for Fog , Maritime Distress Alerts, Heavy ship traffic, and Historic trade routes for wind .

Fog:
At first we thought we could predict foggy areas with airtemp, seasurfacetemp, humidity and dewpoint, but then I read your article:

18th Conference on Numerical Weather Prediction, AMS, 25-29 June 2007, Park City, UT
Numerical Forecast of Fog – Central Solutions
Binbin Zhou1,2 , Jun Du1,2 , Brad Ferrier1,2, Jeff McQueen2 and Geoff Dimego2

which confirms that we would have a hard time making dewpoint calculations from average conditions that would be meaningful. So then I started looking for historical databases of recorded fog (and extent), perhaps some methodical recording of all fog events across the world that could be processed into a meaningful fog density map (monthly) which would show %chance of different types of fog, say

light fog >2nm
med fog >1mn and <2mn
heavy fog <1nm

I have not found this information yet, but I understand from your article there there are new satellite methods which record fog, perhaps there is a database somewhere now which would provide this information? We would very much appreciate your help and suggestions.

Heavy Ship Traffic Areas
Perhaps from some historic AIS database. How to measure density of traffic? by Tons of shipping?

Historic Trade winds Routes and Jimmy Cornell Routes
Clipper routes, Historic Sailing Trade routes, Jimmy Cornell Routes need to be compiled.

Marine Distress Alerts (non-false) The other information we are looking for is non-false incidences of Marine Distress Alerts that would provide a monthly view of the average density and type of events across the world.

• COPS - SARSAT -The International Cospas-Sarsat Programme provides accurate, timely, and reliable distress alert and location data to help search and rescue authorities assist persons in distress. I believe the local MCC's all have a database of events, but it is inaccessible as far as I can determine.
• Global Maritime Distress and Safety System (GMDSS)
• EPIRB, DSC, SART

I've contacted Inmarsat because of an article by Brina Mullen "Inmarsat opens up its safety database" , to reduce false alerts.
Brian Mullen details a process for eliminating false Inmarsat calls and describes work with the local MRCC's and their database. Perhaps Inmarsat will provide us some useful information from their database, which will be a portion of the total maritime distress alerts.

If you have any suggestions for us on these other data itmes that would also be appreciated.

I've copied two of the other people who work on the Opencpn open source navigation system and Sean is the author of Climatology plugin.

Sincerely
Rick Gleason

[rgleason]

Gridded Data
USGOADAE
US Global Ocean Data Assimalation Experiment
USGODAE Data Catalog - Summary View


http://www.usgodae.org/links.html

[barnakiel]

I like and use data from NDBC.

It is not climatology proper but rather historical observation data.

Example of data here: http://www.ndbc.noaa.gov/view_text_f...orical/stdmet/

I mostly view it via the browser but all their data is searchable and downloadable.

Cheers,
b.

[rgleason]

ATOVS Sounding Products

The Advanced TIROS Operational Vertical Sounder (ATOVS) consists of High Resolution Infrared Radiation Sounder (HIRS), the Advanced Microwave Sounding Unit-A (AMSU-A) and Microwave Humidity Sounder (MHS) for retrieving temperature and water vapor soundings in all weather conditions (except heavy precipitation).



STAR / SMCD / OPDB - ATOVS Sounding Products


Wind Vectors from Thomas Cagney
Also from Thomas Cagney:
Noaa Wind Vectors 10x15 25km
http://manati.star.nesdis.noaa.gov/d...ASCATData.php/

[PjotrC]

I found the following charts of interest, and would welcome overlaying them on the nav chart, especially the shipping routes. In fact I was drawing approximate extents of heavy traffic areas by simple lines, to keep aware when I can expect more shipping around...

Live Piracy Map

Maritime Routes and Strategic Passages

[rgleason]

PjotrC, I think both of those charts are excellent and Sean would want to have them in all likelihood.

Between you and Thomas's suggestions there are some very good possible additions.

Rick

[rgleason]

Additional Climatology Data Types -Summary to date

Fog: Montly % chance of heavy (<1nm), medium(>1nm<2nm) or light fog, probably from historical records, a Noaa weather scientist, indicated it might be possible to calculate from available data. Experimental GOES Fog Image http://www.orbit.nesdis.noaa.gov/smc...g/LTSTFGNW.GIF

Still looking for a good historical source that is digital.

Cloud (additional):
ATOVS Aerosol Cloud and Radiation Satellites
STAR / SMCD / OPDB - ATOVS Aerosol, Cloud & Radiation Products

Maritime Distress Alerts (non-false): Average density, by month, maybe several types of distress.

• COPS - SARSAT - Mcc's have databases.
• Cops-Sarsat List of Alerts -good
  
• GMDSS Global Maritime Distress and Safety System
• EPIRB, SART, DSC
• Inmarsat - written to - Brian Mullen "Inmarsat opens up its safety database"

Maritime Piracy Reports
Live Piracy Map

Heavy Ship Traffic Areas: Perhaps from some historic AIS database. How to measure density of traffic? by Tons of shipping? Like marinetraffic.com but historic.
Hofstra.edu geotrans Maritime Routes

Historic Trade winds Routes and Jimmy Cornell Routes: Clipper routes, Historic Sailing Trade routes, Jimmy Cornell Routes need to be compiled.

Ocean Surface Winds
Ocean Surface Winds Data Product Description
Satellite-based sensors are capable of systematically providing measurements over the entire globe. Sensors operating at microwave frequencies can make measurements of the ocean surface day and night and under nearly all-weather conditions.Ocean Surface Products currently available include:

• Wind Vector fields derived from QuikSCAT , ASCAT , WindSAT and ERS-2 .
• Wind Speed fields derived from SSM/I

Other Products include:
Rain, Sea Ice, SST and Water vapor.

NOAA Wind Vectors - Advanced Scatterometer ASCAT

The Advanced Scatterometer (ASCAT) Data Products

ATOVS Sounding Products -Advanced TIROS Operational Vertical Sounder (ATOVS) consists of High Resolution Infrared Radiation Sounder (HIRS), the Advanced Microwave Sounding Unit-A (AMSU-A) and Microwave Humidity Sounder (MHS) for retrieving temperature and water vapor soundings in all weather conditions (except heavy precipitation).
STAR / SMCD / OPDB - ATOVS Sounding Products


Other Sources of Climatological Gridded Data

USGOADAE US Global Ocean Data Assimalation Experiment
GODAE: Global Ocean Data Assimilation Experiment - GODAE Project Links

Global Ocean
USGODAE Data Catalog - Summary View

NDBC National Data Bouy Center -historical observation data.
http://www.ndbc.noaa.gov/view_text_f...orical/stdmet/

Any other thoughts / sources of useful data for Climatology?

[rgleason]

Shipping and Trade Routes, Global and Trade winds,

Ahttp://en.wikipedia.org/wiki/AMVER

Power point
AmverArctic.ppt


Hofstra Main Maritime Shipping Routes


Performance of AIS sensors in space (Has image of ship traffic based on s-AIS at bottom)
Space borne AIS satellite constellation has its own challenges due to the fact that AIS is primarily intended for sea-level reception
Ship Density Maps from S-AIS Global ship density maps serve two main purposes:

• improved understanding of global vessel traffic
• indispensable input dataset for satellite AIS mission planning and simulation

Ship density maps are one way of to analyze ship movements and contribute to a better understanding of the global maritime traffic.

Space based AIS data represent a unique data source for producing such a detailed insight in the global maritime traffic pattern, exemplifying the added value of global vessel tracking data delivered by S-AIS for many other applications.


Gobal Shipping RouteIs Image

Winds in the Age of Sail - Wikipedia, the free encyclopedia -Sailing Routes

Global Wind Systems Ilustrative Basic wind patterns

Round the World by Sea 3 basic routes with consideration of factors.

Trade winds - Large-scale weather - meteoScool - meteoblue
Shows trade winds and Sea routes of Europeans to China & Asia.

Google "Sailing Trade Routes" image search
https://www.google.com/search?q=sail...w=1432&bih=829

[rgleason]

Sean would like use to find sources for Swell and Sea State Data, preferably 30 years of data, but anything would help.

Using boat parameters, wave period and sea state there may be a way to account for the effect of waves in route planning.

[rgleason]

NOAA Experimental Lightning Strike service
Lightning Strike Density

Ocean Prediction Center Unified Analysis
Ocean Prediction Center

Ocean Prediction Center on YouTube
https://www.youtube.com/user/NWSOPC

May be good for Weatherfax!
Atlantic Analysis and Nav-tex
Ocean Prediction Center - Atlantic

Pacific Analysis and Nav-tex
Ocean Prediction Center - Pacific

Ocean Products
Ocean Prediction Center - Ocean Products
Analysis, Forecasts, Model Evaluation - Sea Surface Temperature, Gulf Stream, Satellite Derived (GOES).

[Gnomon]

See also
Real-Time Lightning Map :: LightningMaps.org
life lighning for Europe and starting for USA and southern Pacific

[rgleason]

Found this thread from awhile ago. It's pretty incredible where Sean has taken Climatology since then.

After reading this link again I have two thoughts for consideration and discussion, attempting to get better tools for voyages (may be impractical buy you never know):

1. An extemporaneous grib carnivore which devoures the measured gribs by the satellites and consolidates and summarizes them in 1 hr, 2 hr, 3hr, 6 hr, 12 hr, 24 hr data for say the last month. This data could be downloaded and used by Climatology to "influence" in some percentage selected by user, the 30 year data. Mathematically I am not sure how this could be done.

2. The second idea, hope really, is for a way of summarizing historic frontal activity in a type of playback mode, overlaid on climatology wind gribs and pressure, --- aren't I being a little demanding here! Sorry, just thinking about the possibilities
We would need a noaa database of the actual fronts.

Sent from my SCH-I545 using Cruisers Sailing Forum mobile app

[rgleason]

http://www.cruisersforum.com/forums/...ad.php?t=96792
Sorry there is the link to the thread. Note what we've accomplished. I hope you will use Sean's donate button often, he deserves it, particularly if you find his plugin useful.

Sent from my SCH-I545 using Cruisers Sailing Forum mobile app