Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Search this Thread Rate Thread Display Modes
Old 18-01-2018, 12:15   #1
Registered User

Join Date: Nov 2007
Location: Probably in an anchorage or a boatyard..
Boat: Ebbtide 33' steel cutter
Posts: 4,680
Automatic grib downloading

This took a little while instead of doing more deserving items on the jobs list - but it works!

Briefly, the wonderful node-red is available online >
https://fred.sensetecnic.com/

Sort of swiss army knife fr data - take any input you can think of, fiddle with it then send it to anywhere you can think of. Pretty much. So now every 6 hours it sends an email to saildocs, gts the attachment when it's sent back and saves it into dropbox, once with data/time filename and once call 'current. Once the opencpn grib plugin opens the file called 'current', it does so every time you start the program. Always the latest grib automatically. Assuming you can get online to update dropbox anyway, but then if you can't get online downloading a grib ain't going to happen anyway



FRED is all online so will keep running all the time, though the free version needs you to log on once a day to keep it running.

Quick walk through - top left the Inject node is just trigger which sends something once every 6 hours, or every time you press the button on the left. To the right of that the create message function node creates the email text and sets some other things, it will run anytime something comes in to its input on the left from the inject node so once every 6 hours. Then it sends all that to the email node which sends off the email.

Then further down, bottom right is another email node - once a minute it checks for the latest emil and downloads it. THis the goes to a switch node which is set up to just let through emails from query-reply@saildocs.com. Next is a change node, this swaps the attachment around into a place where the drop box node expects to see it. Then there's another function node which renames the file with current date//time & path in dropbox the off it goes to the dropbox node to get uploaded. As well as this archived file, the flow splits off , waits for a few seconds then gets renamed as Current and uploaded. The msg nodes are just for debugging so you cn see what's going on to help figuring out why it doesn't work.
That's that

Node red code below, if you copy that, open up FRED then press CTRL I you can paste the code straight in. Probably won't work straight away, you might need to install the dropbox node which is easy, button on the control panel on the left.
So constant latest gribs plus saved past gribs .

Code:
[
    {
        "id": "17d3d13f.30215f",
        "type": "e-mail",
        "z": "c676bef7.1661e",
        "server": "smtp.gmail.com",
        "port": "465",
        "secure": true,
        "name": "",
        "dname": "send email",
        "x": 650,
        "y": 80,
        "wires": []
    },
    {
        "id": "5802db4f.e61384",
        "type": "e-mail in",
        "z": "c676bef7.1661e",
        "name": "get email",
        "protocol": "IMAP",
        "server": "imap.gmail.com",
        "useSSL": true,
        "port": "993",
        "box": "INBOX",
        "disposition": "Read",
        "repeat": "60",
        "x": 100,
        "y": 440,
        "wires": [
            [
                "711358ba.286ca8"
            ]
        ]
    },
    {
        "id": "bb666145.e08df",
        "type": "dropbox out",
        "z": "c676bef7.1661e",
        "dropbox": "",
        "filename": "",
        "localFilename": "",
        "name": "upload",
        "x": 550,
        "y": 260,
        "wires": []
    },
    {
        "id": "f95cb5a5.e021c8",
        "type": "debug",
        "z": "c676bef7.1661e",
        "name": "",
        "active": true,
        "console": "false",
        "complete": "true",
        "x": 670,
        "y": 300,
        "wires": []
    },
    {
        "id": "2b8cc36a.a0d37c",
        "type": "inject",
        "z": "c676bef7.1661e",
        "name": "",
        "topic": "saildocs",
        "payload": "send GFS:38N,25N,19W,1W|1.0,1.0|0,6,12|WIND,PRESS,APCP,TCDC,GUST",
        "payloadType": "str",
        "repeat": "21600",
        "crontab": "",
        "once": false,
        "x": 150,
        "y": 100,
        "wires": [
            [
                "f7c7a174.2cf3d"
            ]
        ]
    },
    {
        "id": "9519e8a0.2e7418",
        "type": "change",
        "z": "c676bef7.1661e",
        "name": "",
        "rules": [
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "attachments.content",
                "tot": "jsonata"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 280,
        "y": 380,
        "wires": [
            [
                "11fb456.00987bb",
                "8d39fd28.bfb1a"
            ]
        ]
    },
    {
        "id": "79e9a194.8bfcc",
        "type": "debug",
        "z": "c676bef7.1661e",
        "name": "",
        "active": true,
        "console": "false",
        "complete": "true",
        "x": 610,
        "y": 140,
        "wires": []
    },
    {
        "id": "f7c7a174.2cf3d",
        "type": "function",
        "z": "c676bef7.1661e",
        "name": "Create message ",
        "func": "msg.payload = \"send GFS:48N,25N,29W,1W|1.0,1.0|0,6,12|WIND,PRESS,APCP,TCDC,GUST\\n\";\nmsg.to='xxxxx@saildocs.com';\nmsg.from=\"xxxx@gmail.com\"; //needed to set return-path in the header, otherwise that filed is ampty and saildocs will ignore the email\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 390,
        "y": 80,
        "wires": [
            [
                "17d3d13f.30215f",
                "79e9a194.8bfcc"
            ]
        ]
    },
    {
        "id": "711358ba.286ca8",
        "type": "switch",
        "z": "c676bef7.1661e",
        "name": "Filter",
        "property": "from",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "query-reply@saildocs.com",
                "vt": "str"
            }
        ],
        "checkall": "true",
        "outputs": 1,
        "x": 110,
        "y": 380,
        "wires": [
            [
                "9519e8a0.2e7418"
            ]
        ]
    },
    {
        "id": "11fb456.00987bb",
        "type": "debug",
        "z": "c676bef7.1661e",
        "name": "",
        "active": false,
        "console": "false",
        "complete": "true",
        "x": 350,
        "y": 460,
        "wires": []
    },
    {
        "id": "8d39fd28.bfb1a",
        "type": "function",
        "z": "c676bef7.1661e",
        "name": "Add date to filename",
        "func": "var today = new Date();\n\ntoday.setDate(today.getDate() + 1);\n\nvar hour = today.getHours();\nvar day = today.getDate();\nvar month = today.getMonth()+1;\nvar year = today.getFullYear();\nvar folderdate = year +\"-\" + month;\n\n//node.warn(\"folderdate=\" + folderdate);\n//\nvar filedate= hour + '--' + day +'-' + month + '-' + year;\nnode.warn(\"filedate=\" + filedate);\n\n\nmsg.filename=\"Gribs/\" + folderdate +'/'+ filedate + \".grb\";\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 160,
        "y": 300,
        "wires": [
            [
                "bb666145.e08df",
                "f95cb5a5.e021c8",
                "d1f2f6bf.ad33f8"
            ]
        ]
    },
    {
        "id": "80c80a1f.bf06c8",
        "type": "function",
        "z": "c676bef7.1661e",
        "name": "Change filename to current",
        "func": "msg.filename=\"Gribs/\" + \"current.grb\";\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 340,
        "y": 220,
        "wires": [
            [
                "bb666145.e08df"
            ]
        ]
    },
    {
        "id": "d1f2f6bf.ad33f8",
        "type": "delay",
        "z": "c676bef7.1661e",
        "name": "",
        "pauseType": "delay",
        "timeout": "5",
        "timeoutUnits": "seconds",
        "rate": "1",
        "nbRateUnits": "1",
        "rateUnits": "second",
        "randomFirst": "1",
        "randomLast": "5",
        "randomUnits": "seconds",
        "drop": false,
        "x": 120,
        "y": 220,
        "wires": [
            [
                "80c80a1f.bf06c8"
            ]
        ]
    }
]
__________________

conachair is online now   Reply With Quote
Old 19-01-2018, 00:12   #2
Registered User

Join Date: Sep 2012
Location: Baikal
Posts: 418
Re: Automatic grib downloading

Through Microsoft Flow is easier and free
__________________

Baikal is offline   Reply With Quote
Old 19-01-2018, 00:13   #3
Registered User

Join Date: Aug 2016
Posts: 77
Re: Automatic grib downloading

Under Linux with Shell and Crontab much easier
BlackSea is offline   Reply With Quote
Old 19-01-2018, 01:24   #4
Registered User

Join Date: Nov 2007
Location: Probably in an anchorage or a boatyard..
Boat: Ebbtide 33' steel cutter
Posts: 4,680
Re: Automatic grib downloading

Quote:
Originally Posted by Baikal View Post
Through Microsoft Flow is easier and free
Not heard of that one - thanks for mentioning it, can you rename the files with today's date, overwrite files & create folders on dropbox? Quick look at it, seems quite simple but not as powerful as node red.
Can you download web pages? Might be handy for synoptics.


Could you share a flow downloading renamed grib files as an example?
conachair is online now   Reply With Quote
Old 19-01-2018, 01:27   #5
Registered User

Join Date: Nov 2007
Location: Probably in an anchorage or a boatyard..
Boat: Ebbtide 33' steel cutter
Posts: 4,680
Re: Automatic grib downloading

Quote:
Originally Posted by BlackSea View Post
Under Linux with Shell and Crontab much easier
Constantly running online when your computer is turned off?
Maybe you could share a script for others?
conachair is online now   Reply With Quote
Old 19-01-2018, 01:40   #6
Registered User

Join Date: Sep 2012
Location: Baikal
Posts: 418
Re: Automatic grib downloading

Quote:
Originally Posted by conachair View Post
Not heard of that one - thanks for mentioning it, can you rename the files with today's date, overwrite files & create folders on dropbox? Quick look at it, seems quite simple but not as powerful as node red.
Can you download web pages? Might be handy for synoptics.


Could you share a flow downloading renamed grib files as an example?
I subscribed to receive grib files to outlook mail
on a ready-made template Flows configured Saving the mail attachment to the dropbox in the specified folder.
The filename grib is the same as it comes to the mail
Next, the task script removes old grib files from the folder

https://flow.microsoft.com
Baikal is offline   Reply With Quote
Old 19-01-2018, 01:51   #7
Registered User

Join Date: Aug 2016
Posts: 77
Re: Automatic grib downloading

Quote:
Originally Posted by conachair View Post
Constantly running online when your computer is turned off?
Maybe you could share a script for others?
Runs your NodeRed if your Computer turned off?
BlackSea is offline   Reply With Quote
Old 19-01-2018, 02:09   #8
Registered User

Join Date: Nov 2007
Location: Probably in an anchorage or a boatyard..
Boat: Ebbtide 33' steel cutter
Posts: 4,680
Re: Automatic grib downloading

Quote:
Originally Posted by Baikal View Post
I subscribed to receive grib files to outlook mail
on a ready-made template Flows configured Saving the mail attachment to the dropbox in the specified folder.
The filename grib is the same as it comes to the mail
Next, the task script removes old grib files from the folder

https://flow.microsoft.com
If you can rename the file then opencpn will always open the latest grib automatically. I have unlimited google drive space so hopefully will use IFTTT to move the gribs across to there, just cos I can Gets saved in monthly folders.

I forgot you could subscribe to saildocs! Would you need 4 subscriptions? From the emails it looks like each new grib gets posted at about 05:00/11.00/17.00/23.00 UTC.
conachair is online now   Reply With Quote
Old 19-01-2018, 02:11   #9
Registered User

Join Date: Nov 2007
Location: Probably in an anchorage or a boatyard..
Boat: Ebbtide 33' steel cutter
Posts: 4,680
Re: Automatic grib downloading

Quote:
Originally Posted by BlackSea View Post
Runs your NodeRed if your Computer turned off?
Flow above is running online on FRED, no need for your computer to be turned on. Then when you boot up dropbox updates & the latest grib is downloaded.
conachair is online now   Reply With Quote
Old 19-01-2018, 02:12   #10
Registered User

Join Date: Aug 2016
Posts: 77
Re: Automatic grib downloading

Quote:
Originally Posted by BlackSea View Post
Runs your NodeRed if your Computer turned off?
Oh, sorry, I have not seen, that you use external Node instance somewhere in cloud.
Nevertheless, you can realize something easier without nodered in cloud
BlackSea is offline   Reply With Quote
Old 19-01-2018, 02:19   #11
Registered User

Join Date: Sep 2012
Location: Baikal
Posts: 418
Re: Automatic grib downloading

Quote:
Originally Posted by conachair View Post
I forgot you could subscribe to saildocs! Would you need 4 subscriptions?
I have two subscriptions for different locations
for each his Flows, he searches for the name of the subject of the letter where the coordinates are indicated.
Baikal is offline   Reply With Quote
Old 19-01-2018, 02:24   #12
Registered User

Join Date: Nov 2007
Location: Probably in an anchorage or a boatyard..
Boat: Ebbtide 33' steel cutter
Posts: 4,680
Re: Automatic grib downloading

Quote:
Originally Posted by BlackSea View Post
Oh, sorry, I have not seen, that you use external Node instance somewhere in cloud.
Nevertheless, you can realize something easier without nodered in cloud
Apparently so with microsoft flows, though a quick look didn't make it obvious how easy it is, if at all, to rename attachments in different dropbox locations. Which is sort of the whole point, opencpn always opens the latest grib from dropbox.
Downside of node red online is you need to relog on every day fro the free version. I use it to download live/forecast weather data into a database from openweatheronline as well so don't mind paying a few pence a day, slowly moving everything across from the boat computer online
.
I'll leave this dashboard public for a while - does this link work?
https://paddyb.fred.sensetecnic.com/api/ui/#/0
(Edit - just added graphed live pressure data to that page, took about 2 minutes to set up)


Works well for off site data viewing as well, run a raspberry pi onboard sending data out with MQTT - then view boat data graphed online.

Seems to be easier ways to save an email online but haven't come across anything as powerful as node-red yet. Amazing free program.
conachair is online now   Reply With Quote
Old 19-01-2018, 02:28   #13
Registered User

Join Date: Sep 2012
Location: Baikal
Posts: 418
Re: Automatic grib downloading

opencpn opens the last file by the date the file was created and not by the date in its name!
You can call it anything but it will open the one that was the last by date of the file metadata
Baikal is offline   Reply With Quote
Old 19-01-2018, 02:35   #14
Registered User

Join Date: Nov 2007
Location: Probably in an anchorage or a boatyard..
Boat: Ebbtide 33' steel cutter
Posts: 4,680
Re: Automatic grib downloading

Quote:
Originally Posted by Baikal View Post
opencpn opens the last file by the date the file was created and not by the date in its name!
You can call it anything but it will open the one that was the last by date of the file metadata
So it does! Didn't know that - thanks.

So with that if just downloading a grib a day then something like microsoft flow would probably be easier.
__________________

conachair is online now   Reply With Quote
Reply

Tags
loa, rib

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
OCPN, GRIB, under MAC, Grib tools Gilletarom OpenCPN 0 06-06-2017 08:55
Downloading Only the Changed RNCs / ENCs allanpeda OpenCPN 6 29-11-2011 13:48
Problems Downloading Pages from CF donradcliffe Forum Tech Support & Site Help 34 26-07-2011 14:25
Grib Files and Grib Fantasies maxingout General Sailing Forum 4 16-01-2010 14:54

Advertise Here


Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 22:44.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
×

ShowCase vBulletin Plugins by Drive Thru Online, Inc.