Archive for the ‘coding’ Category

760th day at Eden

Friday, July 9th, 2010

We have a new intern at Eden. Tom Crayford studies Software Engineering at Sheffield University and is doing a 2½ month internship with us, with the possibility of becoming an apprentice at the end of it.

In the style of Tom’s two recent blog posts, First day at Eden and Second day at Eden i thought i might also write about my day, since it was a particularly satisfying one. It just so happens to be two years and a month (760 days) since i joined Eden!

Straight after this morning’s stand-up meeting, I had a quick debrief of a meeting we had with a potential new client yesterday evening. Then i helped Todd to write an email. We draft our emails on Google Wave these days, which is often fun. Four of us were editing the email at once, which means we quickly get to something we’re all happy with before sending it.

I went out for a walk with Frances to pay in some cheques and buy some Friday Treats for everyone!

At lunch time some of us had several rounds of TypeRacer in preparation for Learn To Type Correctly Week next week! I think it’s incredible how so many people have got enthusiastic about learning to type well after Corey Haines announced it. If you don’t believe me, check the twitter hashtag, #learn2typewk!

Something quite unusual happened after lunch. We had a message from Richard Knoll at Mercia Cycling Club in trouble and needing help because the site was not responding. Richard had found us on Google after searching for Ruby on Rails. I agreed to look at it with Tom for an hour and see if we could figure out what was wrong. We determined that rubygems and Radiant had been upgraded on the server and we needed to tweak a few configuration files. With help from Chris and Spencer, the site was back within the hour! Chris then helped me to raise an invoice for an hour’s work.

It’s interesting that we can do anything from a fast restore of an unresponsive website, to projects that take several months from start to finish, requiring two or three pairs of developers working at a time.

Tom and i continued to pair together for the rest of the afternoon. We were able to add value to a project to help a charity in Cambodia. I always learn something whenever i pair with someone. Tom is no exception and i very much enjoyed pairing together today. We all have things we can teach each other.

Tonight i think i might learn a bit from the Ruby Kōans, possibly study a bit of SICP, and definitely see who gets evicted from Big Brother!

How to test your node.js app

Sunday, May 23rd, 2010

I’ve wanted to hack on a node.js project for a while, and a new app idea has given me the perfect excuse. My first question was: how do I test this? It’s a fairly new field out there, and there isn’t much help from node.js itself: it’s much more like Rack than a proper framework. So I spent some time coming up with one way to do it.

Disclaimer: I’m not that experienced with Javascript, particularly with the best way to define objects. I’d be grateful for patches to help improve the quality of the code here. I’ve also borrowed heavily from apprentice-us - thanks to Dave and Corey!

Overview

This is what I’ve got so far (the actual app I’m working will remain closed-source for the mo):

Example node.js github project

You probably want to refer to the code whilst reading the rest of this article.

To run the tests, run rake. To start the app, run node app.js (you will need to have node.js installed obviously).

If you install the watchr gem, and run watchr autotest.watchr, you’ll get robust autotest like functionality. I’m liking watchr much better than ZenTest right now.

How it works

The basic premise is to decouple the request/response handler from the server (see app.js, lib/http.js and lib/router.js). The interesting bit is in test/ integration/ user_sees_homepage.js - this then calls the dispatch method directly, passing in dummy Request and Response objects.

Note how I’ve defined the Response object. This allows me currently to write an integration test that looks like this:

router.dispatch(new Request("GET", "/"), new Response(function(headers, data) {
  assert.contains("200", headers['status'])
  assert.contains("Hello, world!", data)
}));

The assert.contains() method is not part of node.js: it’s implemented in test/helper.js.

The reason you need the asserts to be fired in the end() function is that node.js is inherently asynchronous and will finish executing this file whilst waiting for the haml file to load in lib/router.js. Try it yourself: if you put an assert at the bottom of the file it will fire immediately.

Unit tests

The plan is then to define whichever unit tests you need in test/ unit/ (something)_test.js, with the corresponding code in lib/ models/ (something).js. Just run javascript code in here and call methods on assert, and rake will execute it.

Improvements

You could potentially use the Sinatra-like framework Express to define lib/router.js - I’ve handrolled it for the moment. I’m of the opinion that you spot betterrefactorings by handrolling to start with: it could be that express.js isn’t right for my app, and I can’t easily tell yet.

There are a number of javascript testing libraries out there, but at the moment I’m happy with my own handrolled version, which just relies on the ‘assert’ package that node.js provides. There’s nothing to stop you using JSpec or some other javascript testing library: I wanted to keep things simple to start with.

I’m also aware that Cucumber now supports javascript through V8, which is an important step in the right direction. Unfortunately however it doesn’t yet support the commonjs package system, and doesn’t run through node.js but through raw V8. This makes it hard to use with anything but toy examples. Ideally I’ve love to plug Cucumber in in the future, if we can get it to use node.js as the platform somehow.

If you use it for something useful, let me know! I’d be very happy to receive patches and suggestions.

Every Ash Cloud Has A Silver Lining

Wednesday, May 12th, 2010
Enrique and I recently became victims of the ash cloud, and spent rather longer in the US than we anticipated...

I spent a weekend in Chicago last month chatting to some other exceptional companies about the way we all do business. I learnt a huge amount about Eden and how we want to make it better in the future. We were due to leave on Sunday night, but unfortunately the ash cloud had other ideas, and we were stuck in Chicago for a further five days. It wasn’t much fun to be stranded, but when you’re stuck somewhere, you might as well make the most of it…

Whilst on my extended trip, I got to hang out at 8th Light and Obtiva, at the latter for several days. Thanks especially to Obtiva for hosting us for much of the week! Over the ten days I paired with Micah Martin, Dave Hoover, Chad Prye, Doug Bradbury and Eric Smith. I hacked on RSpec2 briefly with David Chelimsky. I went to visit the new offices at Hashrocket Chicago. I spent several days sharing a little suite of rooms at our hotel with Gustin and Enrique, hacking on edash. I wandered about the city, mostly looking up a lot.

In the evenings, I got to meet and have dinner with Corey Haines’ lovely girlfriend Sarah, to hang out with Chad and Joe from Obtiva at a fascinating restaurant, and ate at a vegan place - not normally on my radar, but it was an experience!

These were some of the many highlights of a five day trip that turned into a ten day experience. Of course I’d have preferred that I wasn’t delayed, but when life gives you lemons… you pair :)

Radiating status at Eden

Monday, March 22nd, 2010

Information Radiators are always a good idea for software teams, and I’ve been pondering how best to show project state at Eden Development for a while.

Here are the various iterations we’ve been through:

Iteration 1: Build status messages

We’ve had a continuous integration server running on integrity for several months, and we wanted to make it obvious how we were doing, so we got an old mac mini out and plugged in a big monitor. That way everyone could see whether our builds were passing or failing. We set the mac to come on at 9am and turn off at 6pm in System Preferences, and used Plainview to display full screen.

This worked well, except that our builds don’t fail that often: our current projects have short enough builds that developers can still get away with running all the tests locally.

Iteration 2: Enter the cycling metric_fu graphs

We have metric_fu running on a private site anyway, but the stats weren’t very visible. Wouldn’t it be cool if we could see our code stats publicly across the whole company? So we split the screen into two halves using a frameset:

Our status board

Each of our projects now cycles through the most important pages from the metric_fu library, for each of our live projects. That way, if there’s a big change in the graphs one day, everyone can see that there’s a problem and can dive in and fix it.

Iteration 3: Cramming more stuff in

So far so good, but we had some blank space down the bottom left! So we shoehorned in part of Pairyapp’s interface, so that everyone could see who was working with who.

This was nice, because people can suddenly see who’s working on a task on their own, and then jump in as needed. It stopped me trawling round the office just to find somebody: now I can easily see exactly who’s doing what (picture at the bottom).

Iteration 4: First pass on our own build server dashboard

This worked well, for about ten days…

…until we saw this. Our little solution was immediately not good enough and I set to work on make it shinier.

At about the same time we changed to using CI Joe for building our projects. The way we got that working is detailed here.

CI Joe doesn’t come with an integrated dashboard, so I set to work writing my own. Here’s where I’ve got to so far:

My dashboard app

It’s not open source yet, but I plan to make it so soon. The pictures are of the person or pair who made the last commit.

Iteration 5: The final result

And here’s how our screen looks this morning:

The final result

(sorry about the censorship)

It’s not finished yet, I’ve plenty more plans. Expect another few posts on this in the future.

UPDATE: Added link to CI Joe post and explained pictures on the dashboard app.

Five things I learnt from Corey Haines

Wednesday, March 17th, 2010

Recently I attended QCon and got a chance on the last day to pair with Corey Haines. We worked on a new rails project we’re building with a few friends (that’s the subject of another post). We’d spent a fair amount of time hanging out, but I hadn’t had a chance to sit down and actually code with him. We paired for a couple of hours in the QCon expo area just as everyone was packing up.

Here are a few lessons and some things I picked up.

REALLY learn vim. Watching Corey fire around vim was something else: my brain could barely keep up with where the cursor was sometimes. Sometimes it felt like he’d just moved the cursor to where he wanted it to be through Sheer Power of Thought. I’m no slouch in vim, but was impressed by just how much faster I’ll be able to go someday, as I continue to practice.

resource_controller. formtastic. That is all. These gems take out the legwork of building a thin restful resource-based rails app. You end up with a lot of tests and very little code to worry about. As webapps become more about javascript and the front-end, rails apps are becoming thinner and thinner, and these gems make them really fast to write.

Alias everything. Corey has a few really useful little bash tricks, like:

alias c='script/console'
alias r='rake routes | grep'

..and some others I didn’t catch. They save so much time and are so obvious that later I found myself banging ‘c’ into a console and wondering why it doesn’t work.

The summary of these lessons is another more general one:

Work to remove whatever constrains you from getting the computer to do what you want. We need to ensure that there is as little as possible in the way of getting stuff done. Everything else is yak shaving: slow typing, tool-illiteracy, whatever. Anytime we’re not thinking about the problem, we’re wasting time.

And finally, a meta-lesson:

Extend your pairing gene pool. It’s amazing how much you learn when you pair with someone outside your immediate sphere. Rather like when I first paired with Enrique, I learnt about stuff I would never have heard of otherwise.

I spent two hours working with Corey and it was a pleasure. Sadly we live a few thousand miles apart, but I’m looking forward to remote pairing sessions in the future.

Archivey the Robot

Monday, March 8th, 2010

Following hot on the heels of Pushy, I’ve implemented the companion application Archivey. This will delete all but the last five messages on a wave, excepting the top message. It’s meant to be used in conjunction with Pushy and any other chatty robots to keep the number of messages in a wave down to a manageable level.

Potential other uses would be in a chatting context: you don’t always want to see the complete history of a chat session and this could be a way to hide the noise. Remember that you can always see the complete history by clicking Playback on the wave, so the messages aren’t lost: they’re just archived.

To use, add archiveyrobot@appspot.com to a wave. Be warned, as soon as a new message is added it will merrily start deleting messages, so be careful!

Source code on github. Hope you like it: let me know if you find it useful.

Introducing Pushy – github notifications to google wave

Sunday, March 7th, 2010

I’ve been having a bit of a love affair with Google Wave recently. Like most people I watched the long introductory video, then tried out the sandbox last July and didn’t really get it. I then read this interesting post which spurred me on to try using it for actual work.

Guess what? It works. Our conversations have become more structured and organised. We’re finding that it does help with keeping everything together in one place, and is more ‘alive’ somehow than a traditional wiki.

So I thought: “Wouldn’t it be cool if you could have your github messages popping up in wave?”

So here’s the results of my handiwork: Pushy.

In simple terms, it’s a robot which accepts any form of HTTP post and adds the content as a new message on the wave. It has special handling for github post-receive hooks: it formats them nicely using a gadget.

How to use it

Log on to wave.google.com and add pushyrobot@appspot.com to a new wave. The robot will add a message giving you the URL to post to:

Pushys receive message

Then, when you post to this url (here I’m using curl):

$ curl -d "testing pushy" http://pushyrobot.appspot.com/push/googlewave.com/fjWFoDWkf

It will add the message to the wave:

The message appears

If you’re using the github notifications, simply add the URL verbatim to your project’s service hooks as a Post-Receive hook:

Github service hook configuration page

Click “Test Hook” and the wave will update. Any new commits to this project should now appear.

Here’s what the commit messages for github commits look like:

Github commit message view

Source code

The source code is at github.com/chrismdp/pushy. It’s my first Python project and first App Engine deployment, so be gentle :) I’d welcome forks and patches: especially if you extend the special formatting for other services.

Enjoy! Do let me know if you use it for anything useful.

UPDATE: The wave forum post discussing the robot is here.

UPDATE: Pushy now supports google code’s PostCommitWebHooks and formats them in a similar way to github commits.

Presentation of my apprenticeship task

Sunday, February 21st, 2010

Here is a presentation i made to some of my colleagues on friday explaining my first apprenticeship task, my wiki, how i went about it, the problems i encountered and what i learnt from the whole experience.

See my Apprenticeship task presentation on youtube.

Licky wiki news

Thursday, February 4th, 2010

Just a quick update on licky, my little wiki apprenticeship task i’m doing. I realise that time has gone on and i haven’t talked about it as much as i might, plus i actually have some quite encouraging news to report now!

Last time i wrote, i was just starting to work on Pages and a PageFactory to find them or create them as necessary. Since then i’ve added a persistence strategy which saves wiki pages to the filesystem. I pass the desired directory to the strategy so that test pages can be kept separate from real pages. This was necessary as i wanted to clear out the test directory every time after use and i didn’t want to accidentally delete real pages.

After that little burst of progress i stalled for a long time as i knew i needed to start writing the web interface and i didn’t know where to begin. James gave me a good idea which was to set up a simple Webserver class that would wrap around the web server of my choice. I have chosen WEBrick for the time being, but it shouldn’t be too difficult to swap in another one later, probably with another strategy pattern. James suggested i start by visiting a page and looking for the text ‘Hello world’. I thought this was a good idea and so i started finding out about WEBrick servlets.

With that start i was suddenly able to leap ahead, testing and implementing everything i need so that my wiki is now able to create, show and edit pages through the web interface. I feel as if i am very close now. Possibly the only thing left to do is add a way to link between pages and then i can truly call it a wiki.

Test driving the development of this whole project has been extremely educational. I know i started off getting too involved in writing a test framework, so Enrique pulled me back and made me think of the simplest thing that could possibly work. I began with one method: assert_equal. That was enough in the beginning. As time went on i added two more methods: assert_true and assert_contains. When necessary i added useful debugging information to report failures. At the appropriate time i refactored the testing framework into a module that could be included into multiple test classes.

I know i also started thinking about the web server too early. I started off trying to work with ERB and templates, as i am familiar with Rails, and in the end all i needed was a few servlets. As Enrique made me realise, the most important part of a wiki is not that it runs on a web server. The most important thing is content. Pages, with content, saved to a file system. Having focussed on that and got a real solid, well tested mechanism for saving and retrieving pages, i knew that the WEBrick servlets could rely on the underlying framework, and when the time came, it did not let me down.

Remote trio programming

Wednesday, January 6th, 2010

If pair programming is good, trio programming must be great, right?

Well, normally no, probably not, but today i had a trio programming experience that worked really well. Two of us were new to the codebase and together we learnt a lot from the person who knows the code well.

Of course, we all did not crowd around a single computer. That would not be comfortable at all. Due to the snowfall, most of the edenites were working from home today, so it gave us a great opportunity to find an effective way to pair remotely.

Initially we tried iChat screen sharing, but the screen updates are too slow to be satisfying. Steve had the idea of using GNU Screen so that we could all access the same terminal window via SSH.

Using a VPN to the office network, we were able to tunnel to Steve’s laptop and access the screen. In fact we found it useful to have two screens running: one for editing code in Vim and another screen for running Cucumber features. The exciting thing is, we don’t all have to look at the same screen at the same time: each person is free to switch screens on their own machine without affecting the others.

A Skype conference call completed the experience so that we could talk through our thought processes and learn the details of the codebase.

It was altogether a very profitable day. The snow, instead of being a hindrance today, helped us to find a very effective way of allowing three people to collaborate on a codebase.