Weather Routing

[seandepagnier]

sorry about all the trouble caused with the various files and paths.. it is a bit of a mess I admit.

I really need to look at it, and hopefully fix the plugin. Maybe it shouldn't cache the contour calculations to disk after all, it is just that they can take a while if you have lots of polars on the same boat, with a slow cpu.


As for polar errors... everyone wants to know why it failed, but this can be for one or several reasons. Furthermore, it might fail to explore in a certain direction because of too much wind for the polar, but succeed in the route a different way.

If I simply reported all of the reasons routing failed to progress in any direction, it would also be misleading in many cases, although maybe it would be better than "polar failed"



I would like to work on this plugin more, and I do realize it is possible to do a lot more things:

1) Variable step time, reducing it, if it fails to route, and retrying (easy)
2) Another would be to avoid "dead ends" in a more intelligent way to speed up routing.
3) I know it's possible to make the routings about 10x faster, but this would be a major effort.
4) Deal with sea state, and wave conditions. I want to do this, but I still don't know how, because it requires a lot more than a 2 dimensional polar.

[rgleason]

Dear Sean,

"You've done real good!" This plugin works great and we just need to make the initial configuration easy for new beginners with paths and directories all setup, and that nice simple climatology based routing configuration that we discussed, so that one press of "Compute" and the configuration "Completes".

Then they can explore the PI by making small changes.

Then I have another idea that will help these users "go back" to the Initial Installation. Make a button "Default Config" that just copies the files from the system folder over the files in the user folder.

This would allow new users to make changes, then compute, then when they get frustrated because it is not working, just hit "Default Config" and the orignal files will be copied over to the User folders and they can start out again.
---Of course if they manage to screw up the directory paths, that problem is on them, they have to figure it out themselves!

Thanks for sharing your 4 point long term goals, and its great you've shared them, but this plugin is good as it is now too.
Item 1 might be very helpful, but you are busy with pypilot and other things too.

We're very lucky Stelian took this on and has figured out most of the immediate things that are needed. You guys make a great team and I am most appreciative.

Hope you're in a warm place.
Best as always,
Rick

[rgleason]

Git fetch, git pull and compile your repos Stelian.
Then cleaned up my configuration - run uninstallweatherrouting.exe, then delete/rename programdata\opencpn\plugins\weatherrouting [user directory] and then check opencpn.ini with Find for weather routing entries and delete them and save the opencpn.ini file.

Then installed Weather_Routing plugin, Start Opencpn v4.8.0 and enable the plugin.

Start Climatology [although this is normally unnecessary].

Start Weather_routing, and the four configurations are all setup ready to compute.

Each configuration computed the first time I highlighted it and then picked "Compute"

See the following shots.

[rgleason]

Sean and Stelian's new version is here for Windows users.

https://github.com/rgleason/weather_routing_pi/releases

[SoggyPaws]

You did it again! Posted within 12 hours two DIFFERENT install exe's! But I stuck with stelian's. And it was not easy to find the "Clean... .exe", so I skipped that step.

But I renamed both directories (data... weather_routing and program... weather_routing_pi) and searched my opencpn.ini for all instances of "weather", "routing", and "wx" and cleaned up everything that looked like it belonged to weather routing. There were some old path declarations in there that probably caused the "save" bug I mentioned before.

Then I hand-cleaned my own weatherroutingconfigutation.xml and boat.xml files (I have 3 different boats I'm working with... myself, and 2 other boats that were on passage or contemplating a passage last year when I was working on it).

All my routes complete, and look "reasonable", though I have not looked at them in detail. I even did a 19-day passage that crossed the dateline and the year boundary, and nothing crashed. This passage went off the end of the GRIB file, both physically and time-wise.

Some comments:
1. The weather routing config box has no "minimize" button. It should? I have to move it off screen to make the chart visible. It also doesn't remember from one execution to another the size of the columns on the result screen (would be nice--if only to widen the column proportionally as you drag the window wider).

2. The export button seems to do nothing. I would expect it to pop open a "where do you want to save your gpx file" window, but nothing happens.

3. There is still nothing in my "contours" folder except an empty file named "C". Should there be? When I ran the test boat congurations included with the install, which started out with a very clean line, like this:

<Polar FileName="TWS-0-6-Power.pol" CrossOverPercentage="0" />

they all ended up in this format:

<Polar FileName="C:\ProgramData\opencpn\plugins\weather_r outing\polars\TWS-0-6-Power.pol" CrossOverContours="C:\ProgramData\opencpn\plugins\ weather_routing\contours\C:!?!ProgramData!?!opencp n!?!plugins!?!weather_routing!?!polars!?!TWS-0-6-Power.pol.contours" CrossOverPercentage="0" />

The path name added by the plugin... it was not there before. And note the very funny path for the Contour... probably why I only have a "C" in my contours file.

4. It would be nice to have the Boat File Name included in the column list on the routing page. The average person may not do this (except maybe in the case of a Boat-Power, Boat-Light Wind, and Boat-Heavy Wind)

5. On the "why did it fail" issue...Is there any logging within the module that could be turned on...when you get a "fail", pop the logged text file, so at least there is a hint...

6. When you get to the "export" function... besides being able to export the gpx file to an OpenCPN route... there should be an option to export a CSV that lets you see the routing in tabular form.

I think the biggest effort should be made to make it stable, and user-friendly.

Thanks for all your work in that regard. Having documentation that matches the plugin is very helpful!! Having some test files that work the first time is also extremely helpful in getting my own stuff set up. Bravo Zulu!

Sherry

[seandepagnier]

Originally Posted by SoggyPaws

But I renamed both directories (data... weather_routing and program... weather_routing_pi) and searched my opencpn.ini for all instances of "weather", "routing", and "wx" and cleaned up everything that looked like it belonged to weather routing. There were some old path declarations in there that probably caused the "save" bug I mentioned before.

I'm not sure if I can do anything to deal with the problem from the above description.

Then I hand-cleaned my own weatherroutingconfigutation.xml and boat.xml files (I have 3 different boats I'm working with... myself, and 2 other boats that were on passage or contemplating a passage last year when I was working on it).

All my routes complete, and look "reasonable", though I have not looked at them in detail. I even did a 19-day passage that crossed the dateline and the year boundary, and nothing crashed. This passage went off the end of the GRIB file, both physically and time-wise.

It should work fine, but you cannot cross both the dateline, and the prime meridian on the same route.

Some comments:
1. The weather routing config box has no "minimize" button. It should? I have to move it off screen to make the chart visible. It also doesn't remember from one execution to another the size of the columns on the result screen (would be nice--if only to widen the column proportionally as you drag the window wider).

How would it minimize? Where would it restore to?

Which column? Doesn't it autosize them? Did you manually size them and want it to remember that?

2. The export button seems to do nothing. I would expect it to pop open a "where do you want to save your gpx file" window, but nothing happens.

It should create a track in opencpn. You can find it in the route manager in the tracks tab with the name "weather route". From there you can export it to gpx.

3. There is still nothing in my "contours" folder except an empty file named "C". Should there be? When I ran the test boat congurations included with the install, which started out with a very clean line, like this:

<Polar FileName="TWS-0-6-Power.pol" CrossOverPercentage="0" />

they all ended up in this format:

<Polar FileName="C:\ProgramData\opencpn\plugins\weather_r outing\polars\TWS-0-6-Power.pol" CrossOverContours="C:\ProgramData\opencpn\plugins\ weather_routing\contours\C:!?!ProgramData!?!opencp n!?!plugins!?!weather_routing!?!polars!?!TWS-0-6-Power.pol.contours" CrossOverPercentage="0" />

The path name added by the plugin... it was not there before. And note the very funny path for the Contour... probably why I only have a "C" in my contours file.

clearly completely broken on windows. Maybe it still works, but has to recompute the contours when you restart the plugin?

4. It would be nice to have the Boat File Name included in the column list on the routing page. The average person may not do this (except maybe in the case of a Boat-Power, Boat-Light Wind, and Boat-Heavy Wind)

Great idea, I can easily add that column which will not be enabled by default

5. On the "why did it fail" issue...Is there any logging within the module that could be turned on...when you get a "fail", pop the logged text file, so at least there is a hint...

There is no logging.

6. When you get to the "export" function... besides being able to export the gpx file to an OpenCPN route... there should be an option to export a CSV that lets you see the routing in tabular form.

I think the biggest effort should be made to make it stable, and user-friendly.

You made it crash the program? How?

As for user friendly.. how, besides better fail messages?

Thanks for all your work in that regard. Having documentation that matches the plugin is very helpful!! Having some test files that work the first time is also extremely helpful in getting my own stuff set up. Bravo Zulu!

Sherry

[did-g]

Originally Posted by boat_alexandra

As for user friendly.. how, besides better fail messages?

Fewer knobs, weather routing should automagically figure out most of them. Maybe a simple and expert dialogs?
Simple dialog could be:
- start/end waypoints. start date (W should figure time step from grib)
- max wind/max wave .
- polar file, should work with plain csv one polar and multiple polars xml.
- maybe motoring speed if sail speed under x.

A reverse routing would be great, find start time for a given end time.

Moreover many errors are close to shore, time or angular step are too big and weather should reduce them before given up.

[Taarnskov]

I would like to have max gust in the table if it available in the grib.

Regards, Lars

[stelian]

Originally Posted by boat_alexandra

clearly completely broken on windows. Maybe it still works, but has to recompute the contours when you restart the plugin?

Right, I can reproduce this on Windows. Works on Linux though.

I'll try to see if I can fix this (just in case my changes introduced this) before submitting the pull request.

[stelian]

Fixed the windows contours problem in: http://popies.net/tmp/weather_routin...7sp1-win32.exe

[stelian]

Pull request submitted !

(Rick, meanwhile, pay attention that I rewrote my github branch in order to cleanup the commits, so a simple pull will give you errors).

[rgleason]

2. The export button seems to do nothing. I would expect it to pop open a "where do you want to save your gpx file" window, but nothing happens.

Try RM tracks.

[rgleason]

Stelian, will remove my local repos and start again or force it.

[seandepagnier]

Originally Posted by did-g

Fewer knobs, weather routing should automagically figure out most of them. Maybe a simple and expert dialogs?
Simple dialog could be:
- start/end waypoints. start date (W should figure time step from grib)
- max wind/max wave .
- polar file, should work with plain csv one polar and multiple polars xml.
- maybe motoring speed if sail speed under x.

So.. re-arrange the configuration gui then? It already hides advanced features in "advanced" and "additional" tabs, but maybe it can be nicer by making a tab for the whole window putting the less used features there. A "default" button also to reset the advanced options?


By polar file, do you mean, instead of a boat file, to allow just a single polar file? Right now it requires a boat with a single polar in it.. is this a problem?

As for motoring speed.. as mentioned many times: this can be built into the polar, or a separate motoring polar used.

If a motoring polar is used, then it has a lot of flexibility as you can specify which angles, and wind speeds to switch over, and the graph shows exactly when the switch occurs. Eventually the report will be able to show even more details, like how long spend on each polar. Also motoring speed will vary depending on conditions as well. So having a simplistic minimum speed like qtvlm is both redundant, limiting and unrealistic. Maybe you would like a button in the boat dialog to generate a motoring polar easily? I will accept pull requests.

Don't expect me to implement this feature, as I strongly discourage anyone from motoring, ever. It is unfair that so many people needlessly and selfishly consume excessive resources, while unique islands wash away, and coral reefs die. I never used an engine, and I've sailed 35,000 miles, including many harbors, reefs, difficult passes and canals) ...

So.. submit pull requests, but we must ensure that motoring is not suggested, used by default, or encouraged.

You might find me in the future implementing calculations for sails shading solar panels in light winds, used for electric propulsion, and "solar tacking" to ensure maximum sunlight throughout the day, including cloud cover data from gribs. Also hydro generators charging batteries, to be used later when sailing is difficult. Already the plugin supports anchoring in adverse currents as this can actually be the fastest route provided the current changes.

A reverse routing would be great, find start time for a given end time.

What, if anything does this actually achieve? I read that it predicts how stable or dependable the route is, but is that actually proven?

Moreover many errors are close to shore, time or angular step are too big and weather should reduce them before given up.

So.. Should it just cut them in half and try again if the routing fails? Maybe cutting angular step down, but not more than to 3 degrees?

I checked the "polar failed" code again. It occurs when there are no polars that specify the boat speed at the current conditions anywhere in the map.

This is misleading, because the polar might not be specified somewhere, resulting in this error displayed, but the real reason the route fails is because there is land in the way, or adverse currents, or some other reason.

Polar failed, or other messages are really only hints, and might not be the actual reason of failure, they just mean that some potential routes being explored were blocked by this. If I reported "landfall detected" then this would also, always be reported if the route touched land anywhere, regardless if this was actually the reason the route failed.

We should work on specific cases, of where "polar failed" or other messages are displayed, and it is misleading or confusing on how to correct the error, and what it should actually display in that particular case. I can try to improve the error messages, but it's not possible to universally provide the most useful error message without knowing the user's intentions (reading their mind)

Originally Posted by Taarnskov

I would like to have max gust in the table if it available in the grib.

Regards, Lars

So this is the maximum wind gust anywhere along the route I assume? I suppose the plot can also plot the wind gust along the route as well. It should be not difficult I will try to implement this.

[rgleason]

We have already learned what happens when a plugin is fully featured but is too complicated for inexperienced people to use. Sean still has the skeleton Velocity Prediction Program VPP in this plugin but the code is buried for the time being. This code would create a boat polar from the boat's basic design specs, but it was complicating the main goal, which is routing. I believe Sean has moved it to some a remote area of the plugin for the time being.

Did-g please explain, (at least to me, what "fewer knobs" means. Is this an effort to simplify the controls, so the user is not confronted with so many choices?) I think so.

I will say that I hardly ever touch "Additional" Tab and this Tab could have a "Default Reset" button at the bottom. However I've found new faster routes by increasing the "Max Search Angle", as Sean suggested.
Also I find it useful to add Tacking time, and to have control of Max latitude on a routing to England when the ice came down last summer, and the Wind vs Current looks very interesting to me as that is where we get into trouble sometimes (I am not sure how it works yet.) But what I am trying to say, is these settings are very useful sometimes, don't use them when starting out! Leave them on the default settings please!

Perhaps the time step from grib_pi would be good, but Weather_Routing can and does interpolate the grib steps, so I would not like to loose that and would prefer it stay this way if possible.

If you look, I did create a polar file Power in the current version. Sean really does not agree with this mode of transportation because it uses up our precious resources, has caused the planet to overheat, and created irreparable harm to our environment. The current power.pol is setup to adjust boat speed based on upwind and downwind and to engage at less than something like 4 knots of wind. I had included it simply to illustrate how to create one.
(Also I had thought that use of this plugin would help powerboats use less fuel if they are smart about their activities, using power with the current and the wind.) However, because Sean has requested that there be no Power.pol file for the plugin, I am going to remove it. Our sense of time & work has changed from earlier days, and a new economy will have to re calibrate these senses. Ideally economies with become less "growth" dependent to be based on "sustainability".

Originally Posted by did-g

Fewer knobs, weather routing should automatically figure out most of them. Maybe a simple and expert dialogs?
Simple dialog could be:
- start/end way points. start date (W should figure time step from grib)
- max wind/max wave .
- polar file, should work with plain csv one polar and multiple polars xml.
- maybe motoring speed if sail speed under x.

Sean wrote:

So.. re-arrange the configuration gui then? It already hides advanced features in "advanced" and "additional" tabs, but maybe it can be nicer by making a tab for the whole window putting the less used features there. A "default" button also to reset the advanced options?

I think specific purposeful moves/changes that are discussed and carefully considered make sense. Sean has already made two major revisions to this Gui, and it needs some careful refinement now.
To put another twist on this, how about a "Simple" TAB?

I do wonder if it would help to move the "Courses relative to true wind" [From, to, by] to be under "Constraints" Could we consider that a "constraint" to? I hardly ever change it, but I find it is not helpful to know that that is what the algorithm is using.

Sean wrote:

By polar file, do you mean, instead of a boat file, to allow just a single polar file? Right now it requires a boat with a single polar in it.. is this a problem?

Sean wrote:

As for motoring speed.. as mentioned many times: this can be built into the polar, or a separate motoring polar used.

If a motoring polar is used, then it has a lot of flexibility as you can specify which angles, and wind speeds to switch over, and the graph shows exactly when the switch occurs. Eventually the report will be able to show even more details, like how long spend on each polar. Also motoring speed will vary depending on conditions as well. So having a simplistic minimum speed like qtvlm is both redundant, limiting and unrealistic. Maybe you would like a button in the boat dialog to generate a motoring polar easily? I will accept pull requests.

Don't expect me to implement this feature, as I strongly discourage anyone from motoring, ever. It is unfair that so many people needlessly and selfishly consume excessive resources, while unique islands wash away, and coral reefs die. I never used an engine, and I've sailed 35,000 miles, including many harbors, reefs, difficult passes and canals) ...

So.. submit pull requests, but we must ensure that motoring is not suggested, used by default, or encouraged.

All good points Sean, and I agree with you, and have marveled at your accomplishments. I will be removing the power polar file from the downloadable plugin. Sorry about that.

Sean wrote:

..calculations for sails shading solar panels in light winds, used for electric propulsion, and "solar tacking" to ensure maximum sunlight throughout the day, including cloud cover data from gribs. Also hydro generators charging batteries, to be used later when sailing is difficult. Already the plugin supports anchoring in adverse currents as this can actually be the fastest route provided the current changes.

I'm interested in all of these, in particular the hydro power, but I hope it can be done in such a way that the plugin does not get too complicated.
Also I still think your VPP has great merit for a new user who wants to get a polar that begins to represent his boat performance. It would only be used occasionally to create some polars to use with wxrte.

Sean Regarding Polar:Fail

Moreover many errors are close to shore, time or angular step are too big and weather should reduce them before given up.
So.. Should it just cut them in half and try again if the routing fails? Maybe cutting angular step down, but not more than to 3 degrees?

I checked the "polar failed" code again. It occurs when there are no polars that specify the boat speed at the current conditions anywhere in the map.

Sean, if this is indeed the case, then reducing the Time Step as I have shown a number of times in posts here and in the wiki, would not allow the "configuration" to "complete", but they do complete. -I'll have to look at this more carefully.

Sean wrote:

This is misleading, because the polar might not be specified somewhere, resulting in this error displayed, but the real reason the route fails is because there is land in the way, or adverse currents, or some other reason.

Often the start or end is close to land and the time step is too large, preventing a successful routing, particularly if the constraint of grib duration is relaxed by use of climatology data to extend the duration of the voyage.
Sean wrote:

Polar failed, or other messages are really only hints, and might not be the actual reason of failure, they just mean that some potential routes being explored were blocked by this. If I reported "landfall detected" then this would also, always be reported if the route touched land anywhere, regardless if this was actually the reason the route failed.

It is very clear that detecting & writing appropriate failure messages is a challenge.

We should work on specific cases, of where "polar failed" or other messages are displayed, and it is misleading or confusing on how to correct the error, and what it should actually display in that particular case. I can try to improve the error messages, but it's not possible to universally provide the most useful error message without knowing the user's intentions (reading their mind)

We need to dig into this, understand the causes of the failures each time and learn. Then we will be able to sort out what to say in the message and what meaningful detectoins are needed!


[/QUOTE]Originally Posted by Taarnskov
I would like to have max gust in the table if it available in the grib.[/QUOTE]

So this is the maximum wind gust anywhere along the route I assume? I suppose the plot can also plot the wind gust along the route as well. It should be not difficult I will try to implement this.

Some better interpretive tools after a routing has been completed:
1. Port & starboard tacks along the route (by color).
2. Graphical colored - wind speed and/or boat speed along the route.
3. Something like the Grib summary view, but a WeatherRouting summary view.
Perhaps we can find some good examples and post screenshots for this.

Let's keep the discussion up. It is an excellent plugin but it can be made better, no doubt.