<< eutony.net

Update 11/10/12
Twitter has switched off the non-versioned API endpoints, which means urls of the form
http://twitter.com/statuses/user_timeline
no longer work. Instead, the URLS have to be
https://api.twitter.com/1.1/statusus/user_timeline.
This change was slated for July this year, and I should have done it months ago! Anyway, The code below has been updated accordingly. Please see this post on the Twitter development blog for more info, including full details of the retired endpoints.

Update 05/11/10
Twitter has switched status ID generator to 'Snowflake', which means IDs are now 64 bit unsigned integers. PHP's JSON parser can't handle > 53 bits, so I had to change the robot to use [id_str] instead. The changes are highlighted in red in the code extracts. Please see this post on Twitter-development-talk for more info.

Update 04/09/10
Twitter has now deprecated basic auth, so you have to use OAuth. Shadowfax had a couple of issues with these instruction on CentOS, and has posted how he did it, so it might be worth a look there if these aren't working for you.

Largely for fun, but partly because Twitter is going to has deprecated basic auth, I decided it was time to learn how to do OAuth.

My sources for this page are a combination of the following:

The main motivation for this is that I have a little Twitter robot (or twibot) that

This robot has evolved from a simple shell script, to a Perl app, to a combination shell script and C libcurl app, and now a PHP script. The common factor has been the use of basic auth...

The first bit to sort out is the OAuth bit. There's masses of stuff on the web on how OAuth works - the important bits are:

  1. Your need to be register an app (your bot) with Twitter, and be supplied with a Consumer Token, which needs to be stored safely.
  2. Your bot needs to use this consumer token to contact twitter, and obtain a Request Token.
  3. This request token is then used in conjunction with a twitter account to obtain an Access token, which needs to be stored safely. (At this point the request token is discarded).
  4. The access token is used by the bot to posts tweets 'as' the twitter account used in step 3.

It is worth noting that with the signing method Twitter uses, each token us make up of two files, a key file and a secret file.

Clear? :)

Don't worry - I'll go through each stage in turn, with code. (Code and commandline extracts are shown in white boxes. The two actual PHP files are shown in white boxes with black text and black border.)

Step 1 - Register an application

  1. Go to https://dev.twitter.com/apps
  2. Create a new application. This process has changed since I last did it, but if offered the choice you want a Client application type, and Read & Write (assuming you want to post updates).

You are now shown an Application Details page, which gives you your consumer key and consumer secret. These need to be available to your bot, but must be kept secret from everyone else.

I do this by having a simple PHP include file that sets up a couple of variables:

config.php <?php $consumer_key = 'xxxxxxxxxxxxx' ; $consumer_secret = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' ; ?>

Step 2 - Get a request token

We now use the consumer token to get a request token (which we later trade in for an access token for a specific account).

First of all, set up PHP with the OAuth extension. On Fedora, this looks like this:

# yum install php-devel # yum install libcurl-devel # pecl install -R /usr/lib/php oauth-0.99.9 Add extension=oauth.so to php.ini, or create and add it to /etc/php.d/oauth.ini

Now we need some PHP to connect to twitter and get the request token:

oauth.php <?php include './config.php'; $oauth = new OAuth($consumer_key,$consumer_secret, OAUTH_SIG_METHOD_HMACSHA1,OAUTH_AUTH_TYPE_URI); $request_token_info = $oauth->getRequestToken('https://api.twitter.com/oauth/request_token'); $request_token = $request_token_info['oauth_token']; $request_token_secret = $request_token_info['oauth_token_secret']; // Save the token to a local file. file_put_contents("request_token", $request_token); file_put_contents("request_token_secret", $request_token_secret); // Generate a request link and output it echo 'https://api.twitter.com/oauth/authorize?oauth_token='.$request_token."\n"; ?>

If we run this, we will see a link that we need to copy and paste into a browser, that will allow us to activate the request token into an access token.

> php oauth.php https://api.twitter.com/oauth/authorize?oauth_token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx >

Do this, and twitter will prompt you for a log in. Login the account that you wish your bot to post to. Twitter will then give you an activation PIN which you need for the next step...

Step 3 - Get an access token

We now need to fire a request off to twitter for an access token, for which we use the code below:

oauth2.php <?php include './config.php'; $oauth = new OAuth($consumer_key,$consumer_secret, OAUTH_SIG_METHOD_HMACSHA1,OAUTH_AUTH_TYPE_URI); $pinFromTwitter = $_SERVER['argv'][1] ; $request_token = file_get_contents("request_token"); $request_token_secret = file_get_contents("request_token_secret"); $oauth->setToken($request_token, $request_token_secret); $access_token_info = $oauth->getAccessToken('https://api.twitter.com/oauth/access_token',null,$pinFromTwitter) ; $access_token = $access_token_info['oauth_token']; $access_token_secret = $access_token_info['oauth_token_secret']; // Now store the access token into another file (or database or whatever): file_put_contents("access_token", $access_token); file_put_contents("access_token_secret", $access_token_secret); // Give it a whirl. // use this access token to verify our crudentials. $oauth->setToken($access_token, $access_token_secret) ; $oauth->fetch('https://api.twitter.com/1.1/account/verify_credentials.json'); $json = json_decode($oauth->getLastResponse()); echo "Access token saved! Authorized as @".(string)$json->screen_name."\n"; ?>

If we run this, you will get an access token for the specific account.

> php oauth2.php 1234567890 Access token saved! Authorized as @whoeveryouare >

On a housekeeping note, I rolled up both these activities into a single file called oauth.php, as follows:

<?php include './config.php'; if ($_SERVER['argc'] == 1) { echo "Usage: ".$_SERVER['argv'][0]." [register | validate <pin>]\n" ; die() ; } $oauth = new OAuth($consumer_key,$consumer_secret, OAUTH_SIG_METHOD_HMACSHA1,OAUTH_AUTH_TYPE_URI); $oauth->enableDebug(); // This will generate debug output in your error_log try { if ($_SERVER['argv'][1] == 'register') { // Generate request token and redirect user to Twitter to authorize $request_token_info = $oauth->getRequestToken('https://api.twitter.com/oauth/request_token'); $request_token = $request_token_info['oauth_token']; $request_token_secret = $request_token_info['oauth_token_secret']; file_put_contents("request_token", $request_token); file_put_contents("request_token_secret", $request_token_secret); // Generate a request link and output it echo 'https://api.twitter.com/oauth/authorize?oauth_token='.$request_token."\n"; exit(); } else if ($_SERVER['argv'][1] == 'validate') { if ($_SERVER['argc'] < 3) { echo "Usage: ".$_SERVER['argv'][0]." validate <pin>\n" ; die() ; } $request_token = file_get_contents("request_token"); $request_token_secret = file_get_contents("request_token_secret"); // Get and store an access token $oauth->setToken($request_token, $request_token_secret); $access_token_info = $oauth->getAccessToken('https://api.twitter.com/oauth/access_token',null,$_SERVER['argv'][2]) ; $access_token = $access_token_info['oauth_token']; $access_token_secret = $access_token_info['oauth_token_secret']; // Now store the two tokens into another file (or database or whatever): file_put_contents("access_token", $access_token); file_put_contents("access_token_secret", $access_token_secret); $oauth->setToken($access_token, $access_token_secret) ; $oauth->fetch('https://api.twitter.com/1.1/account/verify_credentials.json'); $json = json_decode($oauth->getLastResponse()); echo "Access token saved! Authorized as @".(string)$json->screen_name."\n"; } } catch(OAuthException $E) { print_r($E); } ?>

With the following usage:

> php oauth.php Usage: oauth.php [register | validate <pin>] > php oauth.php register https://twitter.com/oauth/authorize?oauth_token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx > php oauth.php validate 1234567890 Access token saved! Authorized as @whoeveryouare >

Step 4 - The Twibot

Steps 1, 2, and 3 are one-time operations (unless the access token expires, or you want to use a different account) - and now for the easy bit.

The robot itself is a simple PHP script called periodically (via cron, for example), that does whatever magic you want it to do.

A sample robot that will post a very simple "Oauth test" as the status update is below. The only 'gotcha' here is that the method of the fetch needs to be OAUTH_HTTP_METHOD_POST when submitting updates. Thus with a robot that reads and writes, you would need to switch between fetches that use OAUTH_HTTP_METHOD_GET and OAUTH_HTTP_METHOD_POST.

<?php include './config.php' ; $oauth = new OAuth($consumer_key,$consumer_secret,OAUTH_SIG_METHOD_HMACSHA1); // Read the access tokens $access_token = file_get_contents("access_token"); $access_token_secret = file_get_contents("access_token_secret"); $oauth->setToken($access_token, $access_token_secret) ; $args=array('status'=>'Oauth test') ; try { $oauth->fetch('https://api.twitter.com/1.1/statuses/update.json',$args, OAUTH_HTTP_METHOD_POST); $json = json_decode($oauth->getLastResponse(),true); if(isset($json['id'])) { echo "Tweet sent!"; // Success - output some success page here } else { // It failed print_r($json); } exit; } catch(OAuthException $E) { print_r($E); } ?>

My robot is going to do the two things mentioned at the top - tweet the RSS weather from the beeb, and retweet all posts from @metofficeYorks.

The RSS bit is fairly straightforward. Download and decode the RSS XML, and bounce it back up. My code here is a little bit ugly, as it makes lots of assumptions about the content of the feed, but it's worked for over a year now!

assuming "top-and-tail" as above... // Fetch the RSS feed in XML format, and load into an XML object. $xmlStr = file_get_contents('http://feeds.bbc.co.uk/weather/feeds/rss/5day/world/4542.xml') ; $xml = simplexml_load_string($xmlStr) ; // Use the title of the first item as our tweet. $args=array('status'=>htmlspecialchars($xml->channel->item[0]->title)) ; $oauth->fetch('https://api.twitter.com/1.1/statuses/update.json',$args, OAUTH_HTTP_METHOD_POST ); // See if it stuck. $json = json_decode($oauth->getLastResponse(),true); if(isset($json['id'])) { printf("BBC Update. New id %s\n", $json['id_str']) ; } else { printf("BBC Failed.\n") ; }

The next part is a bit more tricky. We need to get a note of the last status we retweeted. I should use a database for this, but I'm just using the file system.

The only clever bit is that it globs all files ending ".id", and uses the basename as the twitter account to check/retweet, and the content as the previous status id. You also need to reverse the status array to make retweets appear in the same order as they were originally tweeted.

// Find all matching files. foreach (glob("*.id") as $idfile) { // Read the last retweeted status id, and the twitter account we're retweeting. $lastrt = file_get_contents($idfile) ; $twitterid = substr($idfile, 0, -3) ; // If we're successfully retweeted in the past, start with that one. if ($lastrt > 0) $args=array('since_id'=>$lastrt) ; else $args = array() ; // Fetch all the new tweets since the last one we retweeted (or all the // ones that twitter will provide if this is a new account) $oauth->fetch( sprintf('https://api.twitter.com/1.1/statuses/user_timeline/%s.json',$twitterid), $args, OAUTH_HTTP_METHOD_GET); // reverse it to preverse display order. $json = array_reverse(json_decode($oauth->getLastResponse(),true)); // retweet each one. foreach ($json as $status) { if(isset($status['id_str'])) { // do the retweet. // You could add extra logic here, of course, to conditionally retweet based on content. $oauth->fetch( sprintf('https://api.twitter.com/1.1/statuses/retweet/%s.json',$status['id_str']), null, OAUTH_HTTP_METHOD_POST); // See if it stuck. $json2 = json_decode($oauth->getLastResponse(),true); if(isset($json2['id_str'])) { printf("RT @%s %s\n", $twitterid, $json2['id_str']) ; // Update the last status file. if ($json2['id_str'] > $lastrt) file_put_contents($idfile, $json2['id_str']) ; } else print_r($json2) ; } else { // It failed print_r($json); } } }

Now, wrap that all up into a single file, which takes an optional argument telling it whether or not to do the RSS update.

<?php include './config.php' ; try { $oauth = new OAuth($consumer_key,$consumer_secret, OAUTH_SIG_METHOD_HMACSHA1); // Read the access tokens $access_token = file_get_contents("access_token"); $access_token_secret = file_get_contents("access_token_secret"); $oauth->setToken($access_token, $access_token_secret) ; if ($_SERVER['argc'] == 2 && $_SERVER['argv'][1] == 'BBC') { $xmlStr = file_get_contents('http://feeds.bbc.co.uk/weather/feeds/rss/5day/world/4542.xml') ; $xml = simplexml_load_string($xmlStr) ; $args=array('status'=>htmlspecialchars($xml->channel->item[0]->title)) ; $oauth->fetch('https://api.twitter.com/1.1/statuses/update.json',$args, OAUTH_HTTP_METHOD_POST ); $json = json_decode($oauth->getLastResponse(),true); if(isset($json['id'])) { printf("BBC Update. New id %s\n", $json['id_str']) ; } else { printf("BBC Failed.\n") ; } } // regular polling. foreach (glob("*.id") as $idfile) { $lastrt = file_get_contents($idfile) ; $twitterid = substr($idfile, 0, -3) ; if ($lastrt > 0) $args=array('since_id'=>$lastrt) ; else $args = array() ; $oauth->fetch( sprintf('https://api.twitter.com/1.1/statuses/user_timeline/%s.json',$twitterid), $args, OAUTH_HTTP_METHOD_GET); $json = array_reverse(json_decode($oauth->getLastResponse(),true)); foreach ($json as $status) { if(isset($status['id_str'])) { $oauth->fetch( sprintf('https://api.twitter.com/1.1/statuses/retweet/%s.json',$status['id_str']), null, OAUTH_HTTP_METHOD_POST); // Success - output some success page here $json2 = json_decode($oauth->getLastResponse(),true); if(isset($json2['id_str'])) { printf("RT @%s %s\n", $twitterid, $json2['id_str']) ; if ($json2['id_str'] > $lastrt) file_put_contents($idfile, $json2['id_str']) ; } else print_r($json2) ; } else { // It failed print_r($json); } } } } catch(OAuthException $E) { print($E->lastResponse); } ?>

The final bit is to wrap it up into something that can be called from cron, and then adding some logging and error handling. This is left as an exercise for the reader. :)

<< eutony.net