Coding is like gardening...

Archive for the ‘code’ Category

Eden Hack day report

Friday 13th November 2009 was Eden’s first hack day! Following on from the previous success of “Code something cool in 2 hours” we decided to devote a day to each doing whatever we wanted to do. It could be anything at all; the only requirement was that we had to demo something at the end of the day.

Tris and Richard did a multi-user dungeon game using Rails, Javascript and Juggernaut: The Fall. It was impressive and extensible. It was cool that it was already responding to multiple players by around about lunch time! They did a great job on the user interface.

Chris worked on an integration system between multiple contact lists. Big City. The demo example was between Highrise and Google Contacts. The synchronisation didn’t quite work, but we could see its potential.

Enrique and Steve made an ambitious framework for customers to put features into git with scenarios and steps. Gourd. It will link up with projects and iterations, storing the features safely until it is time for them to be automatically inserted into the filesystem for the appropriate iterations.

Todd spent the day learning how to make iPhone applications, and successfully followed a demo app known as “Whoops, i broke my iPhone!” which made us all laugh … at least 3 times! ;)

Spencer did a useful Sinatra app to help people send HTML-formatted emails. Easymail. I think the idea is to write an HTML template, and then use the tool to add in content. It handles inline CSS and multipart correctly without any fuss.

I (aimee) wrote my first Adobe Air application. Pomo. It is something like the OSX pomodoro application. Not having found a suitable Linux equivalent, i thought i’d write my own in Air to make it cross-platform.

I think we all had a great day, learnt a lot of new things, and enjoyed seeing what we’d all managed to produce at the end of the day. Which is your favourite idea? All these projects are open source, so if you’d like to help out, dive in!

Sing Pomodoro takes the stage

Due to various realisations recently, I’m coming back to coding day to day, which is very cool. For some practice, I thought I’d try my hand at a little Sinatra, fully TDD’d, running on heroku for fun.

The result? Sing Pomodoro:

Sing Pomodoro Screenshot

It’s a little app to track just how well your pomodoros are going. It’s not much right now, but it does have a rather useful README on github which explains how to hook up to its API. You can easily use it with Pomodoro.app on the mac, which is how I’m currently using it.

All it currently does is track Pomodoro timings. We could do with some ideas for which statistics would be most helpful to people. Feel free to fork the repository and commit away!

Introducing Jabbersonic

I’ve been working on improving our build process recently and have been inspired by several articles on Extreme Feedback devices.

Specifically, this blog post at Pragmatic Automation caught my attention: I really liked the idea of being able to listen to complex systems such as a project management flow. Wouldn’t it be cool if you could control a varied soundscape with, say, Jabber messages?

So after four hours of hacking late last night, Jabbersonic was born.

It’s pretty easy to get started. If you’re on a mac with iLife installed it should work out of the box:

sudo gem install gosu xmpp4r-simple
git clone git://github.com/ChrisMDP/jabbersonic.git
cd jabbersonic
bin/server  

This will kick off a gosu app which will listen for Jabber messages sent to the provided account and play different sounds corresponding to different events. An example of usage:

Jabbersonic in Action

Jabbersonic in Action

It’s currently set up for a continuous integration system – it should be trivial to make the Hudson Jabber plugin talk to it, and write a tiny API app for Hoptoad for example. It’s not confined to project management though: there’s a simple configuration file which allows you to make it work for pretty much any complex system you might want to model.

For more information, see the README. Opinion is divided here as to whether a soundscape is actually useful for project management or just an irritation: I guess it will mostly depend on the sound design, but we’ve yet to try it out properly.

Let me know if you try it out, or use it for anything useful.

redirect_to :back not reliable

Suppose you want to allow someone to make a comment and then return to the page they were on. An easy way to do this in Rails is to set the CommentsController method to redirect_to :back

def create
  comment = Comment.new(params[:comment])
  if comment.save
    redirect_to :back
  end
end

But beware! The trouble with this is that Firefox has an option to disable referrers, so you can’t rely upon the HTTP_REFERER being set.

So you could set up a session variable when you display your comment form, to later be used in the controller. The current url is found in request.path

<% session[:return_to] = request.path -%>
def create
  comment = Comment.new(params[:comment])
  if comment.save
    redirect_to session[:return_to]
  end
end

Sure, people can turn off session cookies, but if they do that then you have all sorts of other Rails problems to do with authenticity tokens and user authentication, so I think this is a fairly safe method.

Of course, an even better option is to add the comment by AJAX and avoid the need for a page refresh at all … but that can be the topic of a different blog post!

Under the hood: Not-so-basic authentication

Recently I worked on a project that required a single login to access administration options. There was no need for a full-blown RESTful authentication solution – I was advised to “Just use basic auth!”

Rails makes it easy. You probably know the standard example. You put a before_filter :authenticate in the controllers that require it, and set it up in the Application Controller.

def authenticate
  authenticate_or_request_with_http_basic do |user, password|
    user == 'admin' && password == 'pass'
  end
end

It’s all well and good … until you want to add a log out button. The browser stores the successful login credentials in a sort of cookie, and applies them to every page which requests basic authentication. Once you’ve logged in, it’s actually quite hard to make the browser forget you until you quit and restart the browser. It’s hard, but not impossible. If you can force basic authentication to fail, the browser will throw away the credentials.

So the solution is to add a session variable that says “NO SRSLY, LOG ME OUT PLS!” This is the logout action (a destroy method in a Sessions Controller)

def destroy
  session[:logout_requested] = true
  flash[:notice] = "You have logged out successfully"
  redirect_to(root_path)
end

Now for the tricky bit. The way this works is subtle and takes a moment to figure out each time I think about it. We change the authenticate method in the Application Controller so that as well as checking the username and password, it also ensures that this flag has not been set. Meaning we can cause basic authentication to fail when we want it to.

def authenticate
  authenticate_or_request_with_http_basic do |user, password|
    user == 'admin' && password == 'pass' && session[:logout_requested] != true
  end
  session[:logout_requested] = nil
end

Next time our user goes to a page which requires authentication, the browser still provides the correct username and password, but the flag causes the basic authentication to fail. Obviously we then have to clear the flag straight away, otherwise the user would not be able to get back in again even with the correct credentials. The user must type in the correct login name and password again to be able to get back in.

Perhaps we want to know whether the user is logged in or not, so that we know whether to display an edit button. We can set another session variable. Conveniently, authenticate_or_request_with_http_basic returns a boolean value.

def authenticate
  session[:logged_in] = authenticate_or_request_with_http_basic do |user, password|
    user == 'admin' && password == 'pass' && session[:logout_requested] != true
  end
  session[:logout_requested] = nil
end

Remember to set the flag to false when you log out. Also remember that this flag could be true, false or nil so a check in the Application Controller looks like this:

def logged_in?
  session[:logged_in] == true
end

Finally it’s worth noting that the username and password do not have to be hard-coded like this. It’s simple for an example, but don’t think that’s all there is to basic authentication. There’s nothing to stop you comparing against values in a settings table or even doing a user lookup à la RESTful authentication.

def authenticate
  session[:logged_in] = authenticate_or_request_with_http_basic do |email, password|
    user = User.authenticate(email, password)
    if user && session[:logout_requested] != true
      self.current_user = user
      true
    else
      self.current_user = nil
      false
    end
  end
  session[:logout_requested] = nil
end

Thanks to Richard and Tris for their help in figuring out the not-so-basic aspects of basic authentication! :)

Pivotal Tracker: Fantastic app, shame it’s free

I’ve been digging around for a decent agile PM tool for a while, and stumbled across Pivotal Tracker on Twitter a couple of days ago.

Within about thirty minutes of trying it out, I was totally hooked. We’ve rolled our own basic project management tool internally, but we’ve yet to get the time to put proper iterative development support into the app. Pivotal Tracker just works. With a clean UI and some great reporting tools, it’s all I need to run an agile project.

The one big downside is this: it’s completely free.

Wait, you say: surely that’s a good thing? Well, yes and no.

Why great software should cost money

When I pay for an app on the net, I expect a few things:

  • It’s going to be up pretty much all the time.
  • It’s not going to vanish tomorrow.
  • It’s not going to suddenly cost a fortune.

With Pivotal Tracker, I’ve no idea when they’re going to take the software down for maintenance. I’ve seen reports of it going down without notices for 15 minutes here and there, which is fine if it’s a non-critical service. It is however most definitely not fine when my client is trying to comment on features. I also have no guarantee that it’s not going to suddenly disappear into the night, taking all my stories and possibly the project itself with it.

The other issue is that if I’m paying $20/month for it, I’ve a reasonable expectation that it’s not going to cost say $200/month next month. It’s well within Pivotal’s rights to withdraw the free service and charge what they like for the paying version, without any notice.

Are we going to use it anyway? Yes we are: it’s too good to ignore. However, we’ve put in place the following cron job to mitigate the risk:

*/20 * * * * curl -H "X-TrackerToken: " -H
 "Content-type: application/xml" -X GET
 https://www.pivotaltracker.com/services/v2/projects/39382/stories
 > stories-backup.xml

This little piece of magic copies all our stories in XML form to our backup server every 20 minutes, so we’re not screwed if the site vanishes without notice. It would only take us about an hour to import this XML into our old system should we need to. That takes care of the main risk for me.

The conclusions: Some software shouldn’t be free. If you’re using Pivotal Tracker or any other free service in anger, make sure you’ve got a backup plan.

Twitter integration from your Rails app

We’ve learnt a bit about Twitter from writing ykyat.com.

I particularly want to write about how we fetch Twitter user icons. The icons are stored on AWS (Amazon Web Services) and cannot be deduced from the user name. You have to go through the Twitter API to find the image location.

The first thing I tried was the Twitter4R library. This seems to be a very powerful library, and if you were writing a full Twitter client in Ruby you’d definitely want to consider it. It was simple to get the user icons, but the library just felt a little over-the-top for our needs.

Twitter4R seemed to require authentication with every request, and we soon hit the API limit. I realised this is odd because you really don’t need to authenticate to look at the XML or JSON data about a Twitter user. I decided to go back to basics and do it myself. This is the code for parsing the JSON data and picking up the user icon URL:

def icon_url_for_user(username)
  require 'open-uri'
  require 'json'
  buffer = open("http://twitter.com/users/show/#{username}.json").read
  result = JSON.parse(buffer)
  result['profile_image_url']
end

See! Easy!

Most people don’t change their user icon very often, so once we know where to find a user’s icon, we don’t need to ask Twitter for it again for an arbitrary amount of time. A week seems quite sensible. To that end, we created a lookup table in our database to match Twitter user names to their user icon URL. We added an index to the user name column because it acts as the primary key lookup.

When we want to know a user’s icon, we first look up in our table. If we don’t yet have it, or if the updated_at date is more than a week ago, we check with Twitter for the image location. Otherwise we use our cached location.

Fast and easy! :)

How we made #ykyat

Friday 13th February 2009 was going to be just a normal day, as far as we knew. Then Chris arrived at work at 9am and said, “We’re all going to write a new web app today!” The idea for “You know you’re addicted to” had occurred to Chris the previous day, upon realising that checking Twitter before email represents a pretty serious Twitter addiction! Chris thought it might be fun to see what else people were addicted to, and bring the funny anecdotes together in a place where people could rate them and comment upon them.

We had a quick 15-minute meeting where Chris outlined the concept to us and drew some sketches. Tris and I got straight to work on designing the models whilst Chris put together a visual concept. We had lots of discussion about the colour scheme before agreeing on the bright pink. James and Richard were to design the front end, Tris did the backend data manipulating and URL routing. As the person with previous experience in the Twitter API, my part was to get data in from Twitter and send responses out.

By 10:20 we knew this was going to work! Having looked up search.twitter.com for a few select phrases, we discovered that people were indeed twittering about their various addictions!

Data found on Twitter!

Data found on Twitter!

(more…)

Announcing ykyat.com

Screenshot of ykyat.com

Screenshot of ykyat.com

I’ll blog more about this next week, but just wanted to quickly let you know about our new web application, ykyat.com.

It’s a very simple service built on Twitter. It collates classic ‘you know you’re addicted to…’ jokes on a central website, and allows them to be rated so that the funniest are promoted for all to see.

It was built partly for fun, and partly as a grand experiment in rapid application design and development: we designed, built and deployed the entire app from the ground up in 8 hours 42 minutes. We learnt a lot from the process and I’ll be posting a minute by minute diary with some screenshots next week.

Exporting from Rails to Excel

There are various methods for exporting data from Rails to an Excel spreadsheet, some more complicated than others.

One easy way we’ve used a few times is to make use of Excel’s ability to interpret HTML tables as spreadsheet rows and columns. For example, i can set up a helper method to turn a collection of tasks into a table in a simple HTML page:

module TasksHelper
  include ActiveSupport::Inflector

  def generate_xls(tasks)
    output_columns = [:title, :due_date]

    returning String.new do |str|
      str << "
"
      str << "
"
      output_columns.each do |col|
        str << "


"
      end
      str << "

"

      tasks.each do |task|
        str << "
"
          output_columns.each do |col|
            str << "


"
          end
        str << "

"
      end
      str << "
#{humanize(col)}
#{task.send(col)}
" end end end

The environment.rb file is going to have to know about the .xls extension:

Mime::Type.register "application/vnd.ms-excel", :xls

The TasksController can now be taught to respond to .xls and send_data as a file for download:

class TasksController < ApplicationController
  include TasksHelper

  def index
    @tasks = Task.find(:all)

    respond_to do |format|
      format.html # index.html.erb
      format.xls {
        send_data(generate_xls(@tasks),
          :filename => 'all_tasks.xls',
          :type => 'application/vnd.ms-excel')
      }
    end
  end

end

When visiting /tasks.xls we will now be given the option to download all_tasks.xls which Excel will parse quite happily as a spreadsheet.

One important consideration: Excel is not overly fond of Unicode, and Rails by default will send data in UTF-8 format. If the data contains Unicode characters such as curly quotation marks, you will get some weird output like “ in the spreadsheet.

Not to worry! We can use Ruby’s implementation of the iconv API to convert to Excel’s preferred format, ISO-8859-15.

class TasksController < ApplicationController
  include TasksHelper

  def index
    @tasks = Task.find(:all)

    respond_to do |format|
      format.html # index.html.erb
      format.xls {
        require 'iconv'
        converter = Iconv.new('ISO-8859-15//IGNORE//TRANSLIT','UTF-8')
        send_data(converter.iconv(generate_xls(@tasks)),
          :filename => 'all_tasks.xls',
          :type => 'application/vnd.ms-excel')
      }
    end
  end

end

The Unicode curly quotes are transformed into standard quotes, and Excel is happy again!

Acknowledgements and references:

How to export data as CSV
Working with UTF-8 in PDF::Writer and Ruby on Rails
Iconv Ruby class