Welcome to thatlinuxbox.com Friday, March 29 2024 @ 05:58 AM UTC

Release v1.3 of my dailymile_export tool

  • Saturday, June 18 2016 @ 12:48 AM UTC
  • Contributed by:
  • Views: 2,268
Linux, Open Source, and Tech Stuff

Share
  • Google Plus
  • Facebook
  • Twitter
  • Reddit
  • LinkedIn
  • Digg

Announcing v1.3 of my dailymile export tool

In this most recent release of dailymile_export I have updated the Python version to grab extended information for each entry.

The download bundle is available:

https://github.com/danstoner/dailymile_export/releases

The basic usage info of the python script:

$ python dailymile_export_to_tsv.py --help
usage: dailymile_export_to_tsv.py [-h] [-d] [-e] [-g] [-m MAXPAGES] [-w]
                                  USERNAME

Script to download entries from the dailymile API for a particular user into a
tab-delimited file.

positional arguments:
  USERNAME              The dailymile.com username of the account to export.

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Enable debug level logging.
  -e, --extended        Retrieve extended info for each entry. Extended gear
                        includes Effort, Gear, Weather, and Calories. This
                        will SIGNIFICANTLY impact performance since every
                        single entry will require an additional web request
                        (extended data is not available via the API). Posts
                        must not be set to private in dailymile.
  -g, --gpx             Download the .GPX track for each entry.
  -m MAXPAGES, --maxpages MAXPAGES
                        Maximum number of API requests to make (to limit http
                        requests during testing)
  -w, --disablewarnings
                        Disable urllib3 warnings.

 

 

Read below for execution samples and excerpt from the output file.

Full output file samples are available in the "output_files" directory of the project.

The github repo for the project is located:

https://github.com/danstoner/dailymile_export

If my dailymile export tool is useful to you, or you have suggestions or feedback, please submit a github issue.

 

 

Sample test fetching only 2 pages of entries and included extended info:

$ python dailymile_export_to_tsv.py -e -m 2 danstoner
INFO:root:Max Pages = 2
INFO:root:First API Request: http://api.dailymile.com/people/danstoner/entries.json?page=1
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35671975/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35667222/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35664020/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35646080/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35646020/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35641287/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35634952/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35625017/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35621599/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35621592/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35621557/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35610449/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35599226/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35594410/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35587204/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35579619/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35574810/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35567753/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35559001/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35558933/workout_data
INFO:root:Fetching: http://api.dailymile.com/people/danstoner/entries.json?page=2
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35542790/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35538945/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35525964/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35517561/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35517519/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35512945/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35506318/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35500410/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35496586/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35486611/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35478421/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35478396/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35473418/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35461079/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35454399/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35448963/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35433177/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35433161/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35428708/workout_data
INFO:root:Fetching extended info from http://www.dailymile.com/people/danstoner/entries/35428676/workout_data
WARNING:root:Stopped processing. Hit maxpages limit. Use -m to set a larger value.
INFO:root:Total number of entries: 40
INFO:root:Writing to output file: danstoner_dailymile_export_py.20160618202709.51.tsv

 

Sample from the output file:

id url timestamp title activity_type felt duration_seconds distance distance_units description effort_outof_5 gear weather calories
1980554 http://www.dailymile.com/entries/1980554 2010-05-22T16:00:00Z Hogtown Creek Greenway Running injured 1.5 miles Brand new shoes and this is what I get. 165
1980574 http://www.dailymile.com/entries/1980574 2010-05-25T04:00:00Z None Running alright 2.25 miles Incorporated my first barefoot run. Barefoot 248
1980599 http://www.dailymile.com/entries/1980599 2010-05-29T04:00:00Z None Running alright 2.25 miles CALVES!  1.5 mi of this was barefoot. Nike Free Run+ 248
1981894 http://www.dailymile.com/entries/1981894 2010-06-01T15:49:23Z Just ordered some red RunAmoc running moccasins from Soft Star Shoes.
1993425 http://www.dailymile.com/entries/1993425 2010-06-01T16:00:00Z Hogtown Creek Greenway Running blah 2.5 miles Ran barefoot part of the way, some on hot concrete.  Near-instant blisters.  Eagerly waiting for my RunAmocs to arrive from Soft Star Shoes. 276
2010928 http://www.dailymile.com/entries/2010928 2010-06-03T04:00:00Z Hogtown Creek Greenway Running tired 1 miles Easy walk-jog.  Test-driving my new huarache sandals (kit from invisibleshoe.com).  On hard surfaces I could definitely feel the knot under the left foot to the point of being painful.  The right foot was fine.  Probably that shoddy construction (by me!). The cord between my toes was a little uncomfortable.  However, on soft surfaces these were awesome for running over sticks and pine cones.  Traction was good all-around. Huarache Sandals 110
2019075 http://www.dailymile.com/entries/2019075 2010-06-04T15:09:38Z None Walking alright 4 miles Hiking on Bolen Bluff Trail. I saw deer, red-winged blackbirds, some kind of prairie mice, and wild turkey with babies. 215
2032050 http://www.dailymile.com/entries/2032050 2010-06-05T04:00:00Z Hogtown Creek Greenway Running injured 1 miles Ran in my new huarache sandals.  Still some discomfort around the toes, will continue to tweak the lacing.  They definitely force a barefoot style run and I love the extra traction.  I went slow to concentrate on form.  For the first time I felt a glimmer, just a glimmer, of what I'll be able to do if I get in shape.Huarache Sandals 110
2069257 http://www.dailymile.com/entries/2069257 2010-06-08T04:00:00Z Hogtown Creek Greenway Running good 1579 2.5 miles Ran the whole thing in my Nike Frees.  Felt great until my calf went out and I had to gimp the last little bit. Nike Free Run+ 337
Release v1.3 of my dailymile_export tool | 0 comments | Create New Account

The following comments are owned by whomever posted them. This site is not responsible for what they say.


User Functions

Login

Connect

thatlinuxbox.com is the home of Dan Stoner's Personal Blog, Photos, and More (opinions, rants, techno-babble, and possibly a few useful tidbits of knowledge).

Questions or Comments about this site? Contact danstoner _ at _ gmail.com.

RSS Feed for this blog

Other places to find me on the web:

Twitter

LinkedIn

GitHub

Support This Site

If you like something that you find on this site, please consider making a purchase through one of the links below or sending me an item from my Amazon Wish List.


The Clymb


Awesome VPS hosting by Linode.com