I particularly want to write about how we fetch Twitter user icons. The icons are stored on AWS (Amazon Web Services) and cannot be deduced from the user name. You have to go through the Twitter API to find the image location.

The first thing I tried was the Twitter4R library. This seems to be a very powerful library, and if you were writing a full Twitter client in Ruby you’d definitely want to consider it. It was simple to get the user icons, but the library just felt a little over-the-top for our needs.

Twitter4R seemed to require authentication with every request, and we soon hit the API limit. I realised this is odd because you really don’t need to authenticate to look at the XML or JSON data about a Twitter user. I decided to go back to basics and do it myself. This is the code for parsing the JSON data and picking up the user icon URL:

def icon_url_for_user(username)
  require 'open-uri'
  require 'json'
  buffer = open("http://twitter.com/users/show/#{username}.json").read
  result = JSON.parse(buffer)
  result['profile_image_url']
end

See! Easy!

Most people don’t change their user icon very often, so once we know where to find a user’s icon, we don’t need to ask Twitter for it again for an arbitrary amount of time. A week seems quite sensible. To that end, we created a lookup table in our database to match Twitter user names to their user icon URL. We added an index to the user name column because it acts as the primary key lookup.

When we want to know a user’s icon, we first look up in our table. If we don’t yet have it, or if the updated_at date is more than a week ago, we check with Twitter for the image location. Otherwise we use our cached location.

Fast and easy! :)

We had a quick 15-minute meeting where Chris outlined the concept to us and drew some sketches. Tris and I got straight to work on designing the models whilst Chris put together a visual concept. We had lots of discussion about the colour scheme before agreeing on the bright pink. James and Richard were to design the front end, Tris did the backend data manipulating and URL routing. As the person with previous experience in the Twitter API, my part was to get data in from Twitter and send responses out.

By 10:20 we knew this was going to work! Having looked up search.twitter.com for a few select phrases, we discovered that people were indeed twittering about their various addictions!

Data found on Twitter!

As James and Richard were cutting up the design and turning it into HTML and CSS, Chris made a very welcome coffee round.

At about 11:15 we had a major success for the data import, upon discovering that Twitter Search can bring back up to 100 results at a time. We thought it was limited to 15. This meant that we could go back and select much more data.

Lots of data for us to use

Obviously, not all this data is particularly interesting, and some of it is just retweets of other people’s addictions. Tris helped me to tweak the regular expression to filter out unwanted results.

Meanwhile, the front end was coming along nicely:

An early version of the front end

At midday Chris bought the domain ykyat.com. We were going to use a different domain name, but after seeing that ykyat.com was available, we all agreed that it was a much better domain. I created the Twitter account @ykyat and applied the background image and colour scheme to the Twitter page.

There was a mild panic over URL routing – we wanted to give each ykyat a unique tag, but we also wanted a nice short URL to browse by topic. It was decided to use a tilde when viewing by tag. The routing is rather non-standard for Rails; I found it interesting to see how it is done. Kudos to Tris for more regular expression magic! :)

I spent my lunch break sitting outside the office wondering how best to implement the since_id so as not to bother Twitter Search by asking for the same results more than once. I had three ideas in mind, and it was useful to have a break to consider the most efficient way.

At about 14:00 things were all coming together very nicely! Tris had organised a cron job for the import, and I was very excited to see a tweet of mine imported and displayed in the gorgeous UI that James and Richard had crafted!

A tweet imported and displayed

James and Richard turned their attention to the rating system whilst Tris set up the live server on Amazon EC2. My next task was to tweet back an @ reply to the addicted person, with the URL to let them know they had been featured on #ykyat. Thanks to the simplicity of the Twitter API, this didn’t take long …

Ykyat bot responds to the addicted person

Our Git repository was looking quite exciting at about this time!

Everything coming together

I then started to look at bringing in the Twitter user icons. All was going fine until I hit the limit of API requests per hour. We realised that we couldn’t just ask Twitter for the icon each time every page was loaded; we would have to be more pragmatic about it. We put the user icons on hold. It was getting to late afternoon by now and we also decided we didn’t have time for commenting. Although Tris had written the back end to support comments, we were aware that the Twitter authentication needed to be secure and well tested so we didn’t want to rush that bit.

The last part of the afternoon was spent getting it working on the live server and making a few tweaks and polish to the user interface. We were all very excited to see what we had achieved in a day! Chris announced at 17:42 that we were done. We went home. Half an hour later Chris announced it on Twitter:

Chris announces #ykyat on Twitter

We made a few enhancements over the weekend. Chris worked on the navigator, showing popular categories and popular tweeters. I was itching to leave comments, which was very motivating for me, knowing that I had the power to enable it! I wrote the Twitter authentication and commenting system on Sunday evening and it went live on Monday!

Building #ykyat was an exciting and rewarding experience for all of us. It was good to work together as one team when so often we each work on different projects. It was fascinating to see an idea grow into a live web app in one day. I think we all knew that we could do it, but it was great to be able to prove it to the world!

Screenshot of ykyat.com

I’ll blog more about this next week, but just wanted to quickly let you know about our new web application, ykyat.com.

It’s a very simple service built on Twitter. It collates classic ‘you know you’re addicted to…’ jokes on a central website, and allows them to be rated so that the funniest are promoted for all to see.

It was built partly for fun, and partly as a grand experiment in rapid application design and development: we designed, built and deployed the entire app from the ground up in 8 hours 42 minutes. We learnt a lot from the process and I’ll be posting a minute by minute diary with some screenshots next week.