Newsletter
TechAnV Blog
Get updates on security engineering, Rust, eBPF, and DevSecOps. No spam, unsubscribe anytime.
Check your inbox and click the confirmation link to complete your subscription.
Converting no-decimal-point latitudes and longitudes using jq#
I had some data with weird co-ordinates in it:
1{2 "Agency": "F",3 "Address": "S WAYSIDE DR",4 "CrossStreet": "BLK GULF FWY IB",5 "KeyMap": "534C",6 "XCoord": "-95314927",7 "YCoord": "29716075",8 "CombinedResponse": "F",9 "CallTimeOpened": "02/20/2021 22:29",10 "IncidentType": "Ems Event",11 "AlarmLevel": "0",12 "NumberOfUnits": "1",13 "Units": "A023;"14}Note the "XCoord": "-95314927" and "YCoord": "29716075" in there.
I know that this data refers to somewhere in Houston, and a few people on Twitter pointed out that it’s actually really simple: it’s just missing the decimal point!
Here’s the jq recipe I came up with to transform this:
1{2 latitude: (.YCoord | tonumber / 1000000.0),3 longitude: (.XCoord | tonumber / 1000000.0),4 date: (.CallTimeOpened | strptime("%m/%d/%Y %H:%M") | todate),5 key: (.CallTimeOpened + " " + .XCoord + " " + .YCoord )6} + .The + . at the end adds back on the original object.
Running this example on https://www.jqkungfu.com/ gives the following output:
1{2 "Address": "S WAYSIDE DR",3 "Agency": "F",4 "AlarmLevel": "0",5 "CallTimeOpened": "02/20/2021 22:29",6 "CombinedResponse": "F",7 "CrossStreet": "BLK GULF FWY IB",8 "IncidentType": "Ems Event",9 "KeyMap": "534C",10 "NumberOfUnits": "1",11 "Units": "A023;",12 "XCoord": "-95314927",13 "YCoord": "29716075",14 "date": "2021-02-20T22:29:00Z",15 "key": "02/20/2021 22:29 -95314927 29716075",16 "latitude": 29.716075,17 "longitude": -95.31492718}Bonus is this example: I concatenated together a unique primary key for each record in key, and I figured out how to use the strptime() function to parse a date.