Welcome to thatlinuxbox.com Monday, September 16 2019 @ 10:08 AM UTC

Release v1.3 of my dailymile_export tool

  • Saturday, June 18 2016 @ 12:48 AM UTC
  • Contributed by:
  • Views: 1,236
Linux, Open Source, and Tech Stuff

Share
  • Google Plus
  • Facebook
  • Twitter
  • Reddit
  • LinkedIn
  • Digg

Announcing v1.3 of my dailymile export tool

In this most recent release of dailymile_export I have updated the Python version to grab extended information for each entry.

The download bundle is available:

https://github.com/danstoner/dailymile_export/releases

The basic usage info of the python script:

$ python dailymile_export_to_tsv.py --help

usage: dailymile_export_to_tsv.py [-h] [-d] [-e] [-m MAXPAGES] [-w] USERNAME

Script to download entries from the dailymile API for a particular user into a
tab-delimited file.

positional arguments:
  USERNAME              The dailymile.com username of the account to export.

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Enable debug level logging.
  -e, --extended        Retrieve extended info for each entry. Extended gear
                        includes Effort, Gear, Weather, and Calories. Tthis
                        will SIGNIFICANTLY impact performance since every
                        single entry will require an additional web request
                        (extended data is not available via the API). Posts
                        must not be set to private in dailymile.
  -m MAXPAGES, --maxpages MAXPAGES
                        Maximum number of API requests to make (to limit http
                        requests during testing)
  -w, --disablewarnings
                        Disable urllib3 warnings.

 

Read below for execution samples and excerpt from the output file.

Full output file samples are available in the "output_files" directory of the project.

The github repo for the project is located:

https://github.com/danstoner/dailymile_export

If my dailymile export tool is useful to you, or you have suggestions or feedback, please submit a github issue.

 

 

Sample test fetching only 2 pages of entries and included extended info:

$ python dailymile_export_to_tsv.py -e -m 2 danstoner
INFO:root:Max Pages = 2
INFO:root:First API Request: http://api.dailymile.com/people/danstoner/entries.json?page=1
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35671975/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35667222/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35664020/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35646080/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35646020/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35641287/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35634952/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35625017/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35621599/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35621592/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35621557/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35610449/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35599226/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35594410/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35587204/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35579619/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35574810/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35567753/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35559001/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35558933/workout_data
INFO:root:Fetching: http://api.dailymile.com/people/danstoner/entries.json?page=2
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35542790/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35538945/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35525964/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35517561/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35517519/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35512945/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35506318/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35500410/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35496586/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35486611/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35478421/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35478396/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35473418/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35461079/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35454399/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35448963/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35433177/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35433161/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35428708/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35428676/workout_data
WARNING:root:Stopped processing. Hit maxpages limit. Use -m to set a larger value.
INFO:root:Total number of entries: 40
INFO:root:Writing to output file: danstoner_dailymile_export_py.20160618202709.51.tsv

 

Sample from the output file:


$ head -n 10 danstoner_dailymile_export_py.20160618202709.51.tsv
id     url   timestamp    title    activity_type    felt    duration_seconds    distance    distance_units    description effort_outof_5 gear weather calories
35428676     http://www.dailymile.com/entries/35428676    2016-05-09T17:44:33Z    campus        Running        alright        1500       2.5    miles     2 VFF KSO EVO  314
35428708     http://www.dailymile.com/entries/35428708    2016-05-09T17:49:24Z    Stadium workout    Running        alright        1200       1    miles     Abbreviated RMC style stadium workout. 3   251
35433161     http://www.dailymile.com/entries/35433161    2016-05-10T11:31:42Z    warmup and down    Running        alright        3000       5.45    miles     Had some odd pains in my right foot at first. Legs sore from yesterday's workout. 1 Altra Samson  628
35433177     http://www.dailymile.com/entries/35433177    2016-05-10T11:34:27Z    Track - 9 x 300m alternating pace        Running       alright     862 2.15 miles        Track - 9 x 300m alternating pace between 3k and 1 mile pace. Short 100m jog recovery. 300m splits: 65, 58, 61, 56, 61, 55, 58, 53, 57 4 Altra Samson  175
35448963     http://www.dailymile.com/entries/35448963    2016-05-12T18:20:25Z    noonday warm run on campus   Running        alright       3047         6.8 miles        Perceived effort was high due to the noonday heat. Last 3 miles at just over 7 minute per mile pace.    3    VFF KSO EVO sunny   628
35454399     http://www.dailymile.com/entries/35454399    2016-05-13T16:01:55Z    Long on Hawthorne Trail         Running        blah       6960         13.1            miles     Turned into a real slog fest.     3     Altra Instinct 1.5 black     hot sunny    1457
35461079     http://www.dailymile.com/entries/35461079    2016-05-14T21:20:19Z    Bacon Strip 11      Running    alright        5520       11.18     miles            Hot and hilly in the late afternoon sun.     3     Merrell Bare Access         1155
35473418     http://www.dailymile.com/entries/35473418    2016-05-16T18:20:14Z    hill sprints      Running    alright        3420       4.75         miles            10 x hill sprints after easy run    2     VFF SeeYa LS          716
35478396     http://www.dailymile.com/entries/35478396    2016-05-17T14:39:49Z    warm up and down  Running    alright        1200       1.85         miles               2 Altra Samson         251

 

Release v1.3 of my dailymile_export tool | 0 comments | Create New Account

The following comments are owned by whomever posted them. This site is not responsible for what they say.


User Functions

Login

Connect

thatlinuxbox.com is the home of Dan Stoner's Personal Blog, Photos, and More (opinions, rants, techno-babble, and possibly a few useful tidbits of knowledge).

Questions or Comments about this site? Contact danstoner _ at _ gmail.com.

RSS Feed for this blog

Other places to find me on the web:

Twitter

LinkedIn

dailymile

GitHub

Visit Dan Stoner's profile on Pinterest.

Support This Site

If you like something that you find on this site, please consider making a purchase through one of the links below or sending me an item from my Amazon Wish List.


If you can't speak for yourself, your Road ID will.


Altra Zero Drop Footwear
10% Off and Free Shipping!

10% Off and Free Shipping!
(Discounts will become visible during Checkout)


The Clymb


Awesome VPS hosting by Linode.com