twitterScrape

BarraQDA’s twitterScrape provides a simple but powerful way to mine, process and visualise Twitter data. Its key advantage is its ability to harvest data not only through Twitter’s API but by ‘scraping’ the Web interface directly. This allows you to avoid the limitations that Twitter imposes on users of its API, most significantly the 3,200 limit on a user’s tweets, and the one-to-two week searchable history. With twitterScrape, you can retrieve millions of tweets extending as far back as Twitter’s day zero: 21 March 2006:

$ twitterScrape.py -u jack -q "just setting up my twttr"
# twitterScrape
#     outfile=<stdout>
#     user=jack
#     query=just setting up my twttr
user,date,retweets,favorites,text,lang,geo,mentions,hashtags,id,permalink
jack,2016-11-23T03:44:53,3108,5931,just setting up my twttr…again (account suspension was an internal mistake),en,,,,801270269184647168,https://twitter.com/jack/status/801270269184647168
jack,2006-03-21T20:50:14,93550,72689,just setting up my twttr,en,,,,20,https://twitter.com/jack/status/20
,,,,,,,,,,
Possible missing tweets after id: 20 - 2006-03-21T20:50:14