Google location history for trip visualisation

[RedPhoenix]

WARNING: Post is probably only useful for hardcore linux geeks - but I figure it's worth putting the information out there to save someone else a bit of time, if they need to do the same sort of stuff.

The log book is out on the boat at the moment, but I'm attempting to do some trip-related data visualisation. Sadly, I'm currently stuck at a desk, so I don't have access to it.

The good news is that my google location history probably has some of the details I need. It'll miss stuff like wind speed and sea state, but it'll be a reasonable starting point.

One of the possible activity signatures that google's AI tries to assign to a particular journey, is "SAILING". It doesn't always get it right - but combine it with the "BOATING" and "FERRY TRIP" signatures, and it gets it pretty close most of the time. You can download all your location history data via the google 'take out' functionality, and extract it to your local computer - it'll be in a JSON format - one file per month, in year-based directories.

There are a few utilities out on the internet that will probably help you delve into that data - but none of them really did what I needed, and I'm a command-line junkie at heart, so I used the json 'jq' parser combined with a bit of inline perl, to write out the data I'm interested in.

Here's a sample of the raw json data in the google takeout location history dump:

{
    "activitySegment" : {
      "startLocation" : {
        "latitudeE7" : -336484033,
        "longitudeE7" : 1512756417
      },
      "endLocation" : {
        "latitudeE7" : -336450109,
        "longitudeE7" : 1512795935
      },
      "duration" : {
        "startTimestampMs" : "1578111840000",
        "endTimestampMs" : "1578112180000"
      },
      "distance" : 542,
      "activityType" : "SAILING",
      "confidence" : "HIGH",
      "activities" : [ {
        "activityType" : "SAILING",
        "probability" : 0.0
      } ],
      "editConfirmationStatus" : "CONFIRMED"
    }

The date needs to be mangled into something legible (rather than milliseconds since epoch), and the GPS locations are integers rather than floats.

So here's the command line to pull the data out into something vaguely useful (CSV: date/time, lat/lon).

You'll need jq and perl installed.

me@systemname:~/Downloads/Takeout/Location History/Semantic Location History/2020$ cat 2020_JANUARY.json | jq '.timelineObjects[]? | select(.activitySegment.activityType == "SAILING" or .activitySegment.activityType == "BOATING" or .activitySegment.activityType == "IN_FERRY").activitySegment | "\(.startLocation.latitudeE7),\(.startLocation.longitudeE7),\(.endLocation.latitudeE7),\(.endLocation.longitudeE7),\(.duration.startTimestampMs),\(.duration.endTimestampMs)"' | tr -d '"' | perl -e 'use POSIX "strftime"; while($l=<STDIN>){chomp($l);($lat1,$lon1,$lat2,$lon2,$t1,$t2)=split(/,/,$l);$lat1=$lat1/10000000;$lon1=$lon1/10000000;$lat2=$lat2/10000000;$lon2=$lon2/10000000;$t1=strftime "%d-%m-%Y %H:%M:%S", gmtime($t1/1000.0); $t2=strftime "%d-%m-%Y %H:%M:%S", localtime($t2/1000.0); print "$t1,$lat1,$lon1\n$t2,$lat2,$lon2\n";}' | more
04-01-2020 18:52:57,-33.600668,151.3003399
05-01-2020 18:37:42,-32.7808023,152.1813408
05-01-2020 19:05:08,-32.7203257,152.1744496
....

[StuM]

Enquiring minds want to know:
Why would you let Google track you constantly?

[RedPhoenix]

A great question Stu - and maybe beyond the scope of the thread..

.. But the summary is: There's a big difference between google tracking me, and google tracking the ancient phone that sits on the boat and the account associated with it.