Posting to the Twitter API

Just to provide a little bit of context, I'm working on a project that will be similar to secrettweet. Users will be able to post anonymous messages. Those messages will be reviewed manually through the Django admin, and once they're approved, they'll be posted to a Twitter account through their API.

I'd like to be able to trigger the Twitter post via the admin - in other words, when a message is marked 'approved', saving the record kicks off a process that does the post and updates the message record to include the post date, posted=True, and the new status id. But I'm not sure how to do that, so instead I wrote a script that will probably run on a cron (I'll run it manually until I work out any bugs and figure out what kind of traffic I can expect).

I tried a couple of different methods for posting to the API before I settled on the final script. I could have done it with just one line, using os.system() to execute the curl call suggested in the Twitter API docs:

import os
os.system('curl --user "twitter_username:twitter_password" -d status="' + message + '" http://twitter.com/statuses/update.xml')

But I needed access to the status id in the response, and I couldn't figure out how to get to it this way. So I moved on to subprocess():

import subprocess, Popen
mypost = Popen('curl --user "twitter_username:twitter_password" -d status="' + message + '" \
    "http://twitter.com/statuses/update.xml"', stdout=subprocess.PIPE, shell=True)
return_value = mypost.communicate()[0]

Using subprocess(), I got a return_value as an XMl string, kind of a pain to parse but at least I was a step closer. Just as a note, I didn't try the JSON format, so I'm not sure what kind of response it returns - that might have been easier to parse but I may never know now. I decided to keep moving.

I did try using urllib/urllib2.Request and encoding the username/passwd into the url, but kept getting 403's. I realize now that I probably just needed to add basic authentication to the request headers, but it was late and I was tired:

url = 'http://%s:%s@twitter.com/statuses/update.xml' % (username,password)

There are a few good libraries out there - python-twitter, in particular, is the one that kept coming up in searches. Python-twitter is a complete interface that covers all parts of the Twitter API. I'd recommend using it if you're buiding a project that needs to be able to post and get feeds in various formats. But I only need to post, and I didn't see the sense in installing that much code for my simple needs. Besides, I liked the challenge of writing my own.

In the end, I wound up deconstructing python-twitter, boiling it down to almost nothing and adding a few changes of my own. The method I finally settled on uses urllib/urllib2 and a few other common modules for encoding and parsing:

(Full disclosure: After a clue from the lovely and talented Zach Voase, I've wittled it down even further and changed the code here to reflect said wittling.)

    request = urllib2.Request('http://twitter.com/statuses/update.json')
    request.headers['Authorization'] = 'Basic %s' % ( base64.b64encode(twitter_username + ':' + twitter_password),)
    request.data = urllib.urlencode({'status': message})
    response = urllib2.urlopen(request) # The Response

I'm sure I could probably tweek this a little more - I'm not happy until I've refactored down to the absolute fewest lines of code possible. But for now, this'll do. Here's the complete script, including the db connections I'm setting up to grab unposted records and update them with their new status ids on a successful post:

#!/usr/bin/python

import MySQLdb as Database
import base64, urllib, urllib2

db = Database.connect("myhost", "myuser", "mypasswd", "mydbname")
cursor = db.cursor(Database.cursors.DictCursor)

sql = """SELECT * FROM twitter_message WHERE approved=1 AND posted=0"""
cursor.execute(sql)
data = cursor.fetchall()

username = "twitter_username"
password = "twitter_password"

for record in data:
    message = record["message"]
    message = message.replace(',', '')
    post_data = {'status': message}

    request = urllib2.Request('http://twitter.com/statuses/update.json')
    request.headers['Authorization'] = 'Basic %s' % ( base64.b64encode(twitter_username + ':' + twitter_password),)
    request.data = urllib.urlencode({'status': message})
    response = urllib2.urlopen(request) # The Response

    """ the most painful part is hacking up the response to get at the status id """
    a = response.read()
    b = a.split(',') ## convert the whole thing to a list
    c = b[30] ## hope the id is always the 30th item in the list
    d = INT(c[5:]) ## strip off first part of string, leave behind the number
      
    if response.code == 200 and type(d) == int:
        record_id = record["id"]
        sql = """UPDATE twitter_message posted=1, post_date=NOW(), status_id=%s WHERE id=%s""" %(d, record_id)
        cursor.execute(sql)

cursor.close()
db.close()