Welcome to thatlinuxbox.com Thursday, November 21 2024 @ 10:46 AM UTC
Export entries from dailymile with my export tool
- Saturday, February 21 2015 @ 01:43 PM UTC
- Contributed by: Dan Stoner
- Views: 10,997
I have been using dailymile as my fitness training log since 2010. I have now logged over 1500 entries and 6600 miles total. I don't want to lose that information. I would like to be able to do more advanced anlalytics on my personal fitness data than the dailymile web app provides.
The built-in dailymile.com export feature is abysmal. The file generated includes only a small number of fields (date, activity_type, distance, time, felt, elevation_gain). Seriously, they include elevation gain but do not include the name of the workout or the description?
Because of these limitations, I contacted dailymile support. The response was something along the lines of "you could always write your own tool using the API." So I did.
Announcing v1.1 of my dailymile export tool. The current version is written in Python. The software archive can be downloaded from:
https://github.com/danstoner/dailymile_export/releases
The github repo for the project is located:
https://github.com/danstoner/dailymile_export
Here is the basic usage info:
$ python dailymile_export_to_tsv.py -h
usage: dailymile_export_to_tsv.py [-h] [-d] [-g] username
Script to download entries from the dailymile API for a particular user into a
tab-delimited file.
positional arguments:
username The dailymile.com username of the account to export.
optional arguments:
-h, --help show this help message and exit
-d, --debug Enable debug level logging.
-g, --gear Retrieve gear data also.
Here is some sample output of the running script and the generated output file:
$ python dailymile_export_to_tsv.py danstoner
INFO:root:First API Request: https://api.dailymile.com/people/danstoner/entries.json?page=1
INFO:urllib3.connectionpool:Starting new HTTPS connection (1): api.dailymile.com
INFO:root:Fetching: https://api.dailymile.com/people/danstoner/entries.json?page=2
INFO:root:Fetching: https://api.dailymile.com/people/danstoner/entries.json?page=3
INFO:root:Fetching: https://api.dailymile.com/people/danstoner/entries.json?page=4
...
$ ls -latr *.tsv
-rw-rw-r-- 1 dstoner dstoner 96 Feb 20 09:50 danstoner_dailymile_export.20150220095025.27.tsv
$ head -n4
danstoner_dailymile_export.20150220095025.27.tsv
id url timestamp title activity_type felt duration_seconds distance distance_units description
1980554 http://www.dailymile.com/entries/1980554 2010-05-22T16:00:00Z Hogtown Creek Greenway Running injured 1.5 miles Brand new shoes and this is what I get.
1980574 http://www.dailymile.com/entries/1980574 2010-05-25T04:00:00Z None Running alright 2.25 miles Incorporated my first barefoot run.
1980599 http://www.dailymile.com/entries/1980599 2010-05-29T04:00:00Z None Running alright 2.25 miles CALVES! 1.5 mi of this was barefoot.
I have taken my export file and loaded it into MySQL and PostgreSQL. From there, I can query and filter the results.
For example, I can answer the question "How many days did I run 15 miles or longer during the last 3 months of my marathon training?" Note that I can't just query for rows where the distance column is greater than 15 since some days I might have my running broken into 2 or more entries in my running log (warmup is one log entry, track workout is another). Here is my postgres query after loading the data with the pgimport_dailymile_export_tsv.sql helper script (included in the repo).
fitness=> SELECT entry_date::date, sum(distance) AS daily_miles FROM imported_entries WHERE distance IS NOT null GROUP BY 1 HAVING sum(distance) >= 15 AND now()::date - entry_date::date < 90 ORDER BY 1 DESC;
entry_date | daily_miles
------------+-------------
2015-01-29 | 17.7900
2015-01-22 | 20.3100
2015-01-15 | 18.2400
2015-01-11 | 24.0000
2015-01-08 | 17.1700
2015-01-04 | 15.0000
2014-12-23 | 16.5600
2014-12-19 | 17.4200
2014-12-06 | 20.0000
(9 rows)
I have more enhancements planned for the export script. In particular I want to be able to retrieve my gear info (shoe mileage) which will be trickier since this information is not provided in the dailymile API but will probably need to be retrieved via web scraping.
I might also be providing the script in other languages.
If my dailymile export tool is useful to you, or you have suggestions or feedback, please leave a comment here or as a github issue.
The following comments are owned by whomever posted them. This site is not responsible for what they say.