Cruisers Forum
 


Join CruisersForum Today

Reply
 
Thread Tools Rate Thread Display Modes
Old 04-08-2015, 19:47   #1
Registered User

Join Date: Mar 2013
Posts: 20
AIS from the internet

I wanted an internet based AIS source for debugging but couldn't find one so I created a scraper for SCCOOS using phantomjs. The data is from receivers in Southern California only, but should be good for debugging. The scraper script is written using phantomjs:

Code:
var page = require('webpage').create();
var system = require('system');

page.open("http://sccoos.org/data/ais/", function (status) {
    // Check for page load success
    if (status !== "success") {
        console.log("Unable to access webpage");
    } else {
        page.evaluate( function() {
            d = function(msg,ts){ alert(msg); };
            doStart();
        });
    }
});

page.onAlert = function(msg)
{
    system.stdout.writeLine( msg );
};
This simple script dumps the NMEA strings to standard out if run with phantomjs:

Code:
phantomjs <scriptname>
Or you can make it into a simple AIS server on <port> using netcat like so:
Code:
phantomjs <scriptname> | nc -l <port>
I hope this helps someone.
__________________

__________________
skkeith is offline   Reply With Quote
Old 05-08-2015, 21:10   #2
Registered User

Join Date: Mar 2013
Posts: 20
Re: AIS from the internet

yikes, my script got messed up. It should read

Code:
page.open("http://sccoos.org/data/ais/", function (status) {
    if (status !== "success") {
        console.log("Unable to access webpage");
    } else {
        page.evaluate( function() {
                d = function(msg,ts){ alert(msg); };
                doStart();
        });
    }
});

page.onAlert = function(msg)
{
        system.stdout.writeLine( msg );
};
__________________

__________________
skkeith is offline   Reply With Quote
Old 14-11-2015, 09:26   #3
Registered User
 
LeaseOnLife's Avatar

Join Date: Apr 2008
Location: out cruising again
Boat: Sailboat
Posts: 1,046
Re: AIS from the internet

I spent some hours this morning to find a working internet AIS feed, stumbled over SCOOS, couldn't easily make that webpage/feed work, until I found your script.

Your second post is missing the variable declaration.

Code:
var page = require('webpage').create();
var system = require('system');


page.open("http://sccoos.org/data/ais/", function (status) {
    if (status !== "success") {
        console.log("Unable to access webpage");
    } else {
        page.evaluate( function() {
                d = function(msg,ts){ alert(msg); };
                doStart();
        });
    }
});

page.onAlert = function(msg)
{
        system.stdout.writeLine( msg );
};
Thanks !

Dirk
__________________
LeaseOnLife is offline   Reply With Quote
Old 14-11-2015, 10:31   #4
Registered User
 
boat_alexandra's Avatar

Join Date: Aug 2009
Location: Masachusetts
Boat: bristol 27
Posts: 2,803
Re: AIS from the internet

why do I need phantomjs?

When I goto http://sccoos.org/data/aisfunction in a brower I get an error
__________________
boat_alexandra is offline   Reply With Quote
Old 14-11-2015, 11:22   #5
Registered User
 
LeaseOnLife's Avatar

Join Date: Apr 2008
Location: out cruising again
Boat: Sailboat
Posts: 1,046
Re: AIS from the internet

Quote:
Originally Posted by boat_alexandra View Post
why do I need phantomjs?

When I goto http://sccoos.org/data/aisfunction in a brower I get an error
It starts with going to Experimental Real-Time AIS and then hit the button "start streaming". I tried without phantomjs, but it seems a cookie needs to be set from the originating site, then the AIS data start streaming in. For me, phantomjs made it work:

Code:
# phantomjs scoos.ais.feed.js 


RangeError: Array size is not a small enough positive integer.

  http://maps.google.com/maps?file=api&v=2&key=ABQIAAAAageDAhxOGYlkvDJRgZKB9BROgUKOG613LeCMy88xgOT-b1S-9hTwlKr6gcrDvhleGT_b4XbgDASJuw:134
  http://maps.google.com/maps?file=api&v=2&key=ABQIAAAAageDAhxOGYlkvDJRgZKB9BROgUKOG613LeCMy88xgOT-b1S-9hTwlKr6gcrDvhleGT_b4XbgDASJuw:240
  http://maps.google.com/maps?file=api&v=2&key=ABQIAAAAageDAhxOGYlkvDJRgZKB9BROgUKOG613LeCMy88xgOT-b1S-9hTwlKr6gcrDvhleGT_b4XbgDASJuw:143 in F
  http://maps.google.com/maps?file=api&v=2&key=ABQIAAAAageDAhxOGYlkvDJRgZKB9BROgUKOG613LeCMy88xgOT-b1S-9hTwlKr6gcrDvhleGT_b4XbgDASJuw:240
  http://sccoos.org/data/ais/lib/js/gmap/GChartControl.js:173
  http://sccoos.org/data/ais/lib/js/gmap/GChartControl.js:40
  http://sccoos.org/data/ais/lib/js/gmap/GChartControl.js:329
  http://maps.google.com/maps?file=api&v=2&key=ABQIAAAAageDAhxOGYlkvDJRgZKB9BROgUKOG613LeCMy88xgOT-b1S-9hTwlKr6gcrDvhleGT_b4XbgDASJuw:185
!ABVDM,1,1,,A,1815:@01iJoUrhBAEbKuvs880<0<,0*47
!AIVDM,1,1,,A,15MiRfP000GWf1lBeFVJOot>00Ro,0*53
!ABVDM,1,1,,B,15N00S?P3wGRRBlC62I9@gv>0D0`,0*09
!ABVDM,1,1,,A,35Mo;KgP3nGRHdfC5IwPbOv>20Q0,0*52
!AIVDM,1,1,,B,15N00S?P3wGRRBlC62I9@gv>0D0`,0*02
!AIVDM,2,1,0,B,85MwpV1Kf>DtasWlrPQLpVqK`8PJmWckf`A49f2u2ja,0*3D
!AIVDM,2,2,0,B,?1n`@U`aj,0*6B
!AIVDM,1,1,,A,15NFfR?v@0GWbrLBed76rwv<00SA,0*74
!AIVDM,1,1,,A,15OVL@002pGMj<:CPIoc:8r>0d18,0*56
!ABVDM,1,1,,B,15NWJ9PP00GRV=<CCKPb@Ov>20Rt,0*18
!AIVDM,1,1,,A,176rbc002MGT8D0Br3@Et4l>0<1;,0*57
!AIVDM,2,1,1,B,55N=A>426Wb`piE2220MD4@4iE0F22222222221@Apw,0*04
!AIVDM,2,2,1,B,=A5q:0G4hC`12AAkp0hH88888880,2*1C
!ABVDM,1,1,,A,152Ni`OPCcGQL1PC9d1dDat>0@4<,0*04
!AIVDM,1,1,,A,H5NHOK0<tq5DU=@4@u800000000,2*32
!AIVDM,2,1,7,B,55E:R:02;qv?<H<;V20EHE:1E9=Dh62222222216JQ9;M6oe0M33lp0CQiC1,0*0F
!AIVDM,2,2,7,B,Dp888888880,2*14
!AIVDM,1,1,,A,H52HkAlTCBD5l8C123ijkl1`1130,0*32
!AIVDM,1,1,,B,19NS?P@009GTmibB>EjGeVt@06AD,0*43
!AIVDM,1,1,,B,15N:`e0001GRU9tCC=2ksD40084B,0*75
!AIVDM,1,1,,B,B5NHOK0005qoLbTc`40e7wS5kP06,0*48
!AIVDM,1,1,,B,15NO2a?000GWHr6Bf>ma2Q:@2@4F,0*6F
!AIVDM,1,1,,B,B52NS4h00MpLP>To1qJw`d45WP06,0*7C
!ABVDM,1,1,,B,15N0JwgP4sGRuN2CB>2okwv@084K,0*3A
!AIVDM,1,1,,B,15N0JwgP4sGRuN2CB>2okwv@084K,0*31
!ABVDM,1,1,,A,15N0:I0P00oRql0CCgJ;Kgv@2<24,0*50
!AIVDM,1,1,,B,35Mvli0P@GoQh2bCIHH075`<00v0,0*5E
!ABVDM,1,1,,A,15ND0<PP00GRohFCDiGpDgv>0@4O,0*42
!AIVDM,1,1,,A,15N0:I0P00oRql0CCgJ;Kgv@2<24,0*5B
!AIVDM,1,1,,B,19NSH=A2hToS;iPCB2Q4ESN>0L2H,0*02
!ABVDM,1,1,,B,34hC2g1001oS:RvCAb<c0BB60Dg:,0*65
!AIVDM,1,1,,B,15N6GA0000oWGWfBf3JaJTB@0@4a,0*14
__________________
LeaseOnLife is offline   Reply With Quote
Old 14-11-2015, 12:00   #6
Registered User
 
rgleason's Avatar

Join Date: Mar 2012
Location: Boston, MA
Boat: 1981 Bristol 32 Sloop
Posts: 9,320
Re: AIS from the internet

Sean,
You were looking for testing > 200 boats AIS to optimize AIS?
(I just know what Carcode is thinking about this right now, ..but nothing ventured is nothing gained as they say, and these kinds of changes are not always easy. )

Maybe you can get AIS and radar working well on raspberry_pi, is that the goal? Wouldn't that be remarkable.

Sorry I could not fix the icons into smaller bitmap area, it was headed for trouble, I could tell.

Try OpenCPN Beta File Thingie rguser, rgpass

Home/Nmea/Harmut-Netherlands-Markermeer-Wind-AIS.txt
(better to download the zip)

Set the range to 300 mi and you'll get 203 boats. Hartmut recorded this several years ago. I knew it had lots of AIS but I did not realize it was over 200 boats.
The file is 29 mb but if you download the zip

Harmut-Netherlands-Markermeer-Wind-AIS.zip

it is 5.7 mb.

Then to overload the system turn up the speed on VDR Play to 10x or something.
It would be nice to be able to have a slider that would move the current location in the file, so you could easily replay a segment.
__________________
rgleason is offline   Reply With Quote
Old 10-12-2015, 10:29   #7
Registered User

Join Date: Mar 2013
Posts: 20
Post Re: AIS from the internet

Phantomjs is a crazy beastie. The trick of my script is to replace the main display function in the website's javascript intuitively enough called
'function d( msg, ts )'

incidentally ts is a timestamp if you want it. I am just calling 'allert' with the message and I have also overridden the alert function to writeline the text out.
__________________

__________________
skkeith is offline   Reply With Quote
Reply

Tags
ais, internet

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Live AIS Via Internet sailracer OpenCPN 112 16-07-2015 14:34
AIS on OpenCPN via Marine Traffic Internet v10builder1 OpenCPN 8 13-05-2015 06:07
AIS Simulation - Run VDR to play AIS Nmea file rgleason OpenCPN 16 12-01-2013 11:28
AIS with Internet bzs OpenCPN 31 16-11-2011 16:10
Can You Get Radar & AIS Overlay on Laptop ? AIS Transceiver Recommendations ? lunasea.ds Marine Electronics 22 27-12-2010 14:06



Copyright 2002- Social Knowledge, LLC All Rights Reserved.

All times are GMT -7. The time now is 03:37.


Google+
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Social Knowledge Networks
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.

ShowCase vBulletin Plugins by Drive Thru Online, Inc.