Continuous Integration has hit the mainstream. Teams that haven’t the first clue about agility are using tools like CruiseControl or Hudson to automatically build their software on a central server. Unfortunately, like a lot of things by the time Continuous Integration hit the mainstream the message had been somewhat diluted.

Continuous Integration really doesn’t have much to do with tools like CruiseControl. In fact, the earliest XP teams practiced Continuous Integration without the benefit of such a tool. They simply ran the build locally, checked in the code, moved to another machine – the “integration” machine, checked out the code there, and ran the build there. The original practice of Continuous Integration simply required that every member of the team did this frequently (Not actually “continuously”) as often as several times a day.

Because of the tendency to conflate the tools with that original practice, I often refer to that original practice as Run it Somewhere Else which is intention revealing. Continuous Integration today is really a mixture of three important practices:

1. Collective Ownership – the whole team owns the code. Changes that I make have an effect on the rest of the team, and I need to share them as early as possible.
2. Run it Somewhere Else – in order to know if the code I have created integrates successfully I need to check it in, check it out somewhere else, and verify that it runs outside of my development environment.
3. Ubiquitous Automation – this term was introduced by the Pragmatic Programmers. It means that anything which I must do again I should automate so that I am not duplicating effort. It is the practice of Ubiquitous Automation, and not Continuous Integration, that informs the need for a tool like CruiseControl.

Why CruiseControl is Not Enough

Just running the code with the tool is not enough. Sure, we know that the code is checked out and compiled. Hopefully, we also have some tests and running those tests will tell us if the code works. The latter is actually much more important than the former. What we need is an Informative Build.

What is an Informative Build

An Informative Build is a build that tells us what the state of our development is so that we can make an informed decision. We need an informative build, because otherwise Continuous Integration is just a waste of our time.

That’s right, I said Continuous Integration is a waste of time. It is a waste of time, because simply running a build doesn’t help us unless that build can also tell us what we need to do. An Informative Build:

1. Fails when something is wrong, letting us know that our system is broken and we must fix it.
2. When it fails it tells us precisely why it failed so that we know what we have to do to fix it.
3. When nothing is wrong it doesn’t fail. We shouldn’t be wasting cycles chasing down errors due to brittle tests or external dependencies.

In order to have these qualities, our build must:

1. Have lots of tests that thoroughly cover all of the interesting behaviors of all of the modules in our system.
2. Have informative tests. Tests that fail only when a specific behavior is broken, don’t fail because of some external dependency, and tell us exactly what behavior is causing them to fail so that we know what to fix.
3. Be fast. We need to be able to run it very often so that we know fairly immediately when we have a problem.

Therefore, our tests must:

1. Be isolated – focused on a single interesting behavior and decoupled from any dependencies.
2. Be very, very fast – we need a lot of them, and we need them to be collectively fast. Thus, individually they must be very fast.
3. Be comprehensive – they must exercise the behavior that they test well enough that they fail whenever it fails in whatever way it might fail.