Coding is like gardening...

Billing with Integrity (part 4)

How do you think a client would feel if he got this bill?

How do you think a client would feel if he got this bill?

(You may want to read parts one, two and three first).

Given that “fixed price fixed scope” billing is flawed, a natural next step for many small businesses is to try variable billing. This time we’ll look at some of the pros and cons of such an approach.

By “variable billing” I am referring to the practice of billing per hour (or per day) for your services. It’s a pretty common way of charging out time in the web development industry. Going back to the four variables of software development from my previous post, when billing in this way you are effectively fixing quality (or you should be!), and tying time and cost together. Therefore you are allowing scope and time/cost to vary based on the client requirements.

The advantages

On the surface, variable billing appears to be the answer to all our problems. We don’t have to worry about estimates, because the client is paying per hour, so we’ll just take as long as we need to get things done. Changes in the project are handled easily: we just drop what we’re doing and change direction, so we can give the client the features they want. Goodbye to underbid projects completed for free!

So why is the client often unhappy with this sort of arrangement? Why won’t many clients even sign up to it?

The disadvantages

Many of the advantages are to do with transferring risk away from the developer and onto the client. If the developer is freed from the risk associated with getting a feature done, then that means the client is bearing the full brunt of the uncertainty. Clients want a fixed price, so they can compare you to other suppliers. Just doing variable pricing makes the proposal process very difficult.

There’s also the lack of trust. If you’re just setting up a relationship with the client, they don’t know you from Adam: they have no idea whether whether you’re as good as your word. You might say 2 days for a feature, but if the bill comes in for 4 days, they aren’t going to be happy. It only takes one large unexpected bill to wreck a client relationship.

Another main disadvantage is more subtle. With this sort of approach, a client can sometimes micromanage the project, as they’re rightly concerned about getting value for money. Therefore it’s possible to get inundated with feedback, minor tweaks, irrelevant bug reports and requests for status updates. This burns through yet more of the developer’s time, adding to a client’s bill and dragging down project efficiency. Often clients will want a very detailed breakdown of exactly what time was spent on their project, causing more management overheard.

Conclusion: Useful in some circumstances

Variable billing works well when there’s a large amount of goodwill between client and developer. It doesn’t work so well at the beginning of a client relationship. It’s worked well for us on the two extremes of projects we take on – the very short and the very long:

* Software consultancy, where we’ve been providing short term expert advice on an ad hoc basis.
* Tiny changes, where there’s only a day or two of little tweaks to make to a site.
* Open ended long term projects, where we provide development resource over months/years.

Sometimes projects degenerate into variable billing as the trust grows, a client starts believing our estimates, and everything becomes more informal.

Interestingly, on our most successful long term collaboration using this billing method we have a regular monthly budget which we can’t exceed, but we use that up in a variable ‘per hour’ manner. This is strikingly similar to our current chosen method of billing, which I’ll discuss in the next and final post in this series.

Tune in next time for an outline of our current method (you may want to subscribe for the latest updates). If you’re in a hurry for an “answer”, you could always peek at our pricing pages if you want to know how we bill :) Thanks to Kent Beck’s excellent essay on Optional Scope Contracts (PDF) for many of the ideas in this series, and the feedback I’ve received from various people.

Twitter integration from your Rails app

We’ve learnt a bit about Twitter from writing ykyat.com.

I particularly want to write about how we fetch Twitter user icons. The icons are stored on AWS (Amazon Web Services) and cannot be deduced from the user name. You have to go through the Twitter API to find the image location.

The first thing I tried was the Twitter4R library. This seems to be a very powerful library, and if you were writing a full Twitter client in Ruby you’d definitely want to consider it. It was simple to get the user icons, but the library just felt a little over-the-top for our needs.

Twitter4R seemed to require authentication with every request, and we soon hit the API limit. I realised this is odd because you really don’t need to authenticate to look at the XML or JSON data about a Twitter user. I decided to go back to basics and do it myself. This is the code for parsing the JSON data and picking up the user icon URL:

def icon_url_for_user(username)
  require 'open-uri'
  require 'json'
  buffer = open("http://twitter.com/users/show/#{username}.json").read
  result = JSON.parse(buffer)
  result['profile_image_url']
end

See! Easy!

Most people don’t change their user icon very often, so once we know where to find a user’s icon, we don’t need to ask Twitter for it again for an arbitrary amount of time. A week seems quite sensible. To that end, we created a lookup table in our database to match Twitter user names to their user icon URL. We added an index to the user name column because it acts as the primary key lookup.

When we want to know a user’s icon, we first look up in our table. If we don’t yet have it, or if the updated_at date is more than a week ago, we check with Twitter for the image location. Otherwise we use our cached location.

Fast and easy! :)

How we made #ykyat

Friday 13th February 2009 was going to be just a normal day, as far as we knew. Then Chris arrived at work at 9am and said, “We’re all going to write a new web app today!” The idea for “You know you’re addicted to” had occurred to Chris the previous day, upon realising that checking Twitter before email represents a pretty serious Twitter addiction! Chris thought it might be fun to see what else people were addicted to, and bring the funny anecdotes together in a place where people could rate them and comment upon them.

We had a quick 15-minute meeting where Chris outlined the concept to us and drew some sketches. Tris and I got straight to work on designing the models whilst Chris put together a visual concept. We had lots of discussion about the colour scheme before agreeing on the bright pink. James and Richard were to design the front end, Tris did the backend data manipulating and URL routing. As the person with previous experience in the Twitter API, my part was to get data in from Twitter and send responses out.

By 10:20 we knew this was going to work! Having looked up search.twitter.com for a few select phrases, we discovered that people were indeed twittering about their various addictions!

Data found on Twitter!

Data found on Twitter!

Read the rest of this entry »

Announcing Rails Business (UK)

One thing I’ve always thought missing from the Ruby on Rails scene in the UK is a decent get together of Rails business people – those who run their own businesses like myself. There’s plenty of technical groups, or groups that meet in the evening, but I couldn’t find anything in the daytime.

The best way to get what you want is to do it yourself, right? Therefore I’ve created a meetup group for it. The main aim is to discuss such things as: best practices in running a rails consultancy (of any size), best-practice ruby on rails development/project management, this Rails Maturity Model idea that’s been doing the rounds in the last few weeks, plus anything else that’s relevant.

Anyone who’s interested in the same sorts of issues discussed in this group is welcome to attend. For more details of our first meeting on 4th March, see here.

Coffee Perk

Recently, in addition to my usual work, i have been learning how to make an expresso, latte, cappuchino and americano. I now know all about coffee grinding, tamping and how to steam milk!

This is all thanks to our amazing new coffee machine!

Coffee machine Delicious coffee!

Mmmm, nothing like a lovely cup of frothy, hot Eden Development coffee to get me coding on cold and frosty mornings! :)

If you’re coming to visit any time soon, you can look forward to being welcomed with a delicious cup of coffee of your choice! We even have a sofa and a coffee table! :)

Announcing ykyat.com

Screenshot of ykyat.com

Screenshot of ykyat.com

I’ll blog more about this next week, but just wanted to quickly let you know about our new web application, ykyat.com.

It’s a very simple service built on Twitter. It collates classic ‘you know you’re addicted to…’ jokes on a central website, and allows them to be rated so that the funniest are promoted for all to see.

It was built partly for fun, and partly as a grand experiment in rapid application design and development: we designed, built and deployed the entire app from the ground up in 8 hours 42 minutes. We learnt a lot from the process and I’ll be posting a minute by minute diary with some screenshots next week.

Web development is moving on: are you?

Great article on thinkvitamin.com today, although there was one small point I wanted to quibble:

First of all, for the most part it’s impossible to develop offline ‐ this is because we use hosted services, APIs and most likely you’ll write something that uses some data stored elsewhere on the web.

I disagree — if you’re doing things properly you should be stubbing that API in your test and using git locally, allowing you to work anywhere. Testing the integrated whole must usually be done online, but full integration tests can be always be run later when you’ve done your piece of work and you’re back on the grid.

Exporting from Rails to Excel

There are various methods for exporting data from Rails to an Excel spreadsheet, some more complicated than others.

One easy way we’ve used a few times is to make use of Excel’s ability to interpret HTML tables as spreadsheet rows and columns. For example, i can set up a helper method to turn a collection of tasks into a table in a simple HTML page:

module TasksHelper
  include ActiveSupport::Inflector

  def generate_xls(tasks)
    output_columns = [:title, :due_date]

    returning String.new do |str|
      str << "
"
      str << "
"
      output_columns.each do |col|
        str << "


"
      end
      str << "

"

      tasks.each do |task|
        str << "
"
          output_columns.each do |col|
            str << "


"
          end
        str << "

"
      end
      str << "
#{humanize(col)}
#{task.send(col)}
" end end end

The environment.rb file is going to have to know about the .xls extension:

Mime::Type.register "application/vnd.ms-excel", :xls

The TasksController can now be taught to respond to .xls and send_data as a file for download:

class TasksController < ApplicationController
  include TasksHelper

  def index
    @tasks = Task.find(:all)

    respond_to do |format|
      format.html # index.html.erb
      format.xls {
        send_data(generate_xls(@tasks),
          :filename => 'all_tasks.xls',
          :type => 'application/vnd.ms-excel')
      }
    end
  end

end

When visiting /tasks.xls we will now be given the option to download all_tasks.xls which Excel will parse quite happily as a spreadsheet.

One important consideration: Excel is not overly fond of Unicode, and Rails by default will send data in UTF-8 format. If the data contains Unicode characters such as curly quotation marks, you will get some weird output like “ in the spreadsheet.

Not to worry! We can use Ruby’s implementation of the iconv API to convert to Excel’s preferred format, ISO-8859-15.

class TasksController < ApplicationController
  include TasksHelper

  def index
    @tasks = Task.find(:all)

    respond_to do |format|
      format.html # index.html.erb
      format.xls {
        require 'iconv'
        converter = Iconv.new('ISO-8859-15//IGNORE//TRANSLIT','UTF-8')
        send_data(converter.iconv(generate_xls(@tasks)),
          :filename => 'all_tasks.xls',
          :type => 'application/vnd.ms-excel')
      }
    end
  end

end

The Unicode curly quotes are transformed into standard quotes, and Excel is happy again!

Acknowledgements and references:

How to export data as CSV
Working with UTF-8 in PDF::Writer and Ruby on Rails
Iconv Ruby class

Billing with Integrity (part 3)

More likely with fixed price projects...

More likely with fixed price projects...

(You may want to read parts one and two first).

This time we’ll look at four variables of software development which always apply to any software project. We’ll see how they interact with each other, how a fixed price project model is affected by them, and a potential end result which is often a reduction in quality and more bugs.

So what are these four variables of development? Briefly, they are:

  • Scope – the size of the project.
  • Time – the amount of time taken to get a project done
  • Cost/Resources – the amount of money the project is costing
  • Quality – the end quality of the project (i.e. how robust it is)

You can fix two or three of these variables but it isn’t possible to fix all four.

For example: suppose a feature will take four days to do properly. If you want to take two days over it, then you can either:

  • Increase cost/resources (add more developers, not generally a good idea)
  • Reduce scope (cut the requirements for the feature in half)
  • Rush the work (stay up late, or just code too fast) which then litters the code with bugs, reducing quality.

Fixed price projects try and fix all four of these: “I want this spec, by this date, costing this much. Oh, and no bugs.” So when the inevitable happens and project scope changes, what can be done? The cost is already fixed, or only changeable through a long-winded change process, and there may simply be no money to throw at the project. The deadline could be extended (increase time), but that then leaves the developer out of pocket if the cost stays the same, and the client is unlikely to be happy.

Often the only option is that a developer will revert to rushing the work, to try and get it done as quickly as possible. This leaves a project that looks just great on the outside, and may work ok when delivered to the client, but after a few months it becomes obvious that it only just hangs together. The client might request a change that appears simple on the outset, but because the original work was rushed, the work takes much longer than the developer thought and exposes underlying bugs. Automated tests are often skipped when work is rushed, which further reduces confidence in the codebase when something needs to change.

Conclusion: Don’t fix project price and scope

So, in conclusion, a long fixed price project for a fixed spec is counter-productive. As we’ve already seen, it can allow developers to take advantage of clients, it pits the client and developer against each other, and in practice it fosters bugs in software. Change is the natural state of a software project, and by definition, this model is incompatible with change.

Next time we’ll look at variable billing, by charging for small increments of time. We’ll see how this model interacts with the four variables discussed above, and some of the advantages and disadvantages it poses to both clients and developers.

Tune in next time for a discussion on variable billing (you may want to subscribe for the latest updates). If you’re in a hurry for an “answer”, you could always peek at our pricing pages if you want to know how we bill :) Thanks to Kent Beck’s excellent essay on Optional Scope Contracts (PDF) for many of the ideas in this series and to Ngọc Hà for the photo.

Billing with Integrity (part 2)

Should have written a clearer spec...

Should've written a clearer spec...

(You may want to read part one first).

We left off saying that fixed price projects had a number of disadvantages. One other problem with these types of projects is that they’re implicitly set up to pit client and supplier against each other. This wastes valuable project budget on working out what to do when the project changes.

Imagine a client and developer have come to an agreement and spent a lot of time writing the spec for a project. The contract is signed for a developer to deliver the spec for £4,000. Immediately we have two contrasting forces at work:

  • The client is wanting the maximum amount of features that the developer will give him, so he gets the best return on investment.
  • The developer is hoping to spend the minimum amount of time on the project to get the work done, so that he can cram as many projects in and make as much money as possible.

Good developers and good clients will mitigate these forces, but each party will always need to protect their interests to some extent, or they’ll go out of business.

There’s enough tension in the above relationship as it is. What happens when the client wants to add in another couple of features? The client might argue that the new features aren’t really new, but part of the original spec, so that he can try and get as much as he can for his money. The developer is likely to argue that everything that differs from his interpretation of the spec is new, and therefore should be charged for. The result is usually a fight, hopefully a compromise, but always a big waste of time and energy. Who wants to plough through a huge change process just because the original spec wasn’t clear enough? That time could be better spent actually delivering more features and providing more value to the client.

In practice, specs are never clear enough, and always require changing. Therefore at the beginning of the project you are committing to spending a fair amount of time arguing about the spec. If the change is a big one, then you’ve got a big problem: a lot of time has been spent already getting this spec “right”, and now it’ll need to be thrown away or rewritten significantly. There’s also more pressure to rush this new spec, which results in a worse spec the next time, causing more problems down the road.

The net result of all this is less efficient and more costly projects, bad feeling between clients and developers, and an overall reduction in the success of the project. Forget that ‘ongoing high quality relationship’ both sides wanted at the beginning: for serious failures often the only priority is to escape without a lawsuit.

It’s not all doom and gloom: some fixed-price projects do go as planned, but setting up projects in this way makes it all that much harder to see them through. I’m also not against specs: clearly it’s important for the client to communicate what they want! I’ll discuss how we use specs (although we call them something different) in a later post.

So take heart: there are more modern project management methodologies which ensure that there’s that much more chance that projects succeed and that everyone gets along. Next time we’ll look at some fundamental laws of software development in the search for a better way.

Tune in next time for the four variables of software development (you may want to subscribe for the latest updates). If you’re in a hurry for an “answer”, you could always peek at our pricing pages if you want to know how we bill :)