BigVisible Blog

This is something I talk about a lot in my technical training, and I really mean it. There are a couple of things I would like everyone to think about:

First, programming is automating. If you are a programmer what you do is tell computers how to do things for people. That is automation. If you need to do things you should take advantage of the talent you have for automating and automate the things you need to do.

Second, as a programmer your time is expensive. Also, the things you program are complex and the opportunity to make mistakes is ever present. For both of these reasons you don’t want to be doing the same things over and over again. Any time you are going to do the same thing again you should automate it.

I want all the programmers out there to be very suspicious of doing things that are similar to things they have done before. When you write some code that is similar to code you have written before you are creating duplication. You should find that duplication and refactor to eliminate it.

Similarly, when you have to do some activity that you have done before, such as testing for regressions, deploying an application onto a server, uploading data to a database, creating some report, or any of the many administrative tasks you do every day, you should automate it. That way you will spend less time doing it over and over and you will have fewer opportunities to make mistakes.

Faster and higher quality for the small cost of a little programming (Ostensibly what you want to be doing anyway.) How does that sound?

It is sometimes attractive to impose processes. The reasons that it is attractive make sense: We know what works based on our experience. Those of us who study process have a lot of anecdotal evidence that some things are better than other things, and why not share our wisdom with those who lack our wear?

However, this only makes sense if we don’t value learning. If we don’t value learning then we can assert that we who know, know. We know because we have learned, and we can share what we know with those less fortunate than us. They should be grateful to have benefited from our wisdom and to have been spared the many trials that we have endured.

If we value learning then we value systems that can learn, but that implies that we must let go. We must let go of the idea that what we have learned is what others should learn. Their experiences are necessarily different than ours. If they succeed it is their success. If they fail it is not our failure.

The quality of a good coach/teacher/parent, is to provide everything that is needed to succeed, to motivate, to encourage, to endure, and to get the hell out of the way.

There has been a storm of blog posts, tweets, and other noises lately about why Agile is this or that, why it is good or bad, why it works or doesn’t work, etc. None of it actually matters.

There are two things we ought to be talking about: 1) How do we change our organizations to optimize the delivery of value? 2) How do we make the lives of people involved in software development better?

XP, Scrum, Lean, Kanban, Agile, and whatever else are collections of ideas and practices each united by strong principles and all designed to help us figure out how to solve some of these problems. Each is limited by its point of view. None is a panacea. None will fit your situation “out-of-the-box” without considerable thought and adaptation.

So, use these ideas and these practices. Do so thoroughly and unapologetically. But, I don’t care what you think Agile is or is not, whether you think it is good or bad, whether you think it works or doesn’t work, whether you think I am doing it right or doing it wrong. It doesn’t matter.

[In a private email discussion with the other BigVisible coaches I came up with the following metaphor. I thought it might be of general interest, so I'm reproducing it here with a few small edits.]

An Agile team is like an engine. An engine needs four basic things to work well:

1. Feed it lots of fuel and air at the right moment.
2. Ignite the fuel and air at the right moment to generate power.
3. Transfer power to the drive line to do some useful work.
4. Remove waste gasses at the right moment to maintain volumetric efficiency on the next stroke.

Agile teams need similar things:

1. Feed them valuable ideas to implement at the right moment.
2. Implement high-quality software at the right moment to realize value.
3. Transfer that value out to the customer so that they can do their work.
4. Receive feedback at the right moment to maintain the delivery of value on the next iteration.

Extending the metaphor:

• Scrum is a bigger engine. It is theoretically capable of generating higher output, if you give it all of the things above.
• XP (or “technical practices” if you prefer) is a well tuned ignition system. You need it to maximize efficiency and clean power output.
• Kanban says, “Find the bottleneck and eliminate it.” That is great advice (For engines too) but maybe not specific enough to tell us how to make a Civic beat a Ferrari. [Note: I deliberately diminished the importance of finding and eliminating bottlenecks here. To be clear, I think it is one of the most important skills a coach (or engine builder) can have.]
• Effective product ownership and related business practices are your intake plenum. The fastest way to sap an engine’s power is to starve it here (That’s why they call it the “throttle.”)
• Your direct line to your customer is like your exhaust system. The tighter and twistier and more packed with crap it is the less power you will get.

To go fast you need all of these parts to work together. So, start with a bigger engine. Sure, why not? Start with more efficient spark. Sure, why not? If you want to get faster you’ve got to understand the whole system, and it more or less doesn’t matter where you start.

Jerry Weinberg commented on my last post regarding the Definition of Done saying that we aren’t really done until we have stopped. When pressed further he said that even delivering to production was insufficient, because we might get it wrong, and if we get it wrong we aren’t really “done.”

Of course, he’s absolutely right, and it leads nicely to one of my favorite topics that I’ve been wanting to blog about:

Agile software development is about learning. We work from a premise that the final solution is not known (or knowable) until we have delivered it, and that, therefore, we must collaborate with customers to build and define a working solution incrementally. Along the way we learn about the problem space that we are exploring and we incorporate that knowledge and our own expertise to drive towards a heuristic solution.

That may seem obvious, but it has an interesting implication. All software development processes/methodologies tend to obsess over the question of how much we can get done per unit time. However, if we know that we are engaged in learning, and we understand that learning means sometimes getting it wrong in order to adjust and get it right, then how much we can do is the wrong question. The right question is how long does it take before we can find out that we are wrong. In other words, what is the delay before our next opportunity to learn?

Scrum’s notion of “Done” is not really about being done, but about getting to the point where we can get real feedback to learn from. The same is true for the Lean notion of “Cycle Time.” Each cycle is an opportunity to learn. If there is any chance that we will get it wrong then we need at least two cycles. In fact, the likelihood that we will ever get it right is directly related to the number of opportunities we have to learn before we stop. I’ll let you chew on that for a while.

[Note: the assertion in the last paragraph originally said that the likelihood of getting it right within some period of time was inversely related to the length of our learning cycles. Hopefully, the way I wrote it above is somewhat clearer. The notion of keeping learning cycles small is a natural and important consequence of the above. You will hear more from me about that at some point.]

One slightly peculiar question that comes up over and over again with new Scrum teams is: what is the definition of done? And, how do we determine what our team definition of done is?

It is a peculiar question, because if we weren’t practicing Scrum the answer would be obvious. In fact, the Lean and Extreme Programming communities more or less agree on what “done” means without any real discussion: Done == deployed in production, in the hands of real customers.

When Scrum folks talk about the “Definition of Done,” however, they mean something slightly different. There is an important assumption here: software can reach an intermediate state called “done” where it is not yet “released,” but is “potentially shippable.”

Potentially shippable is an equally peculiar idea. In theory it ought to mean that to release or not to release is purely a business decision at this point. The software is complete, accepted, and tested, but it is not released. In practice it often means that the main development activities have been completed but there is some amount of intangible work to be done that may or may not fit inside of a Sprint (e.g. QA testing, deployment to various environments, formal release process, etc.) This is a poor, but common, way to address the “Definition of Done.”

Potentially shippable software is actually a useful idea. From a technical excellence point of view the goal should be continuous delivery, but from a business perspective continuous delivery might not be feasible. In fact, in some domains big-bang delivery actually has inherent value (e.g. any COTS, especially commercial games, and various forms of online media.) [actually, even in those environments continuous delivery is useful, but only after the initial release hits its date...] What we want is for continuous delivery to be attainable, so that it becomes a business decision not to do it. Thus, XP has long had the concept of always releasable software.

So, how do Scrum teams define done? I suppose the real answer is, however they want to, but I suggest the following: Start with everything that it would take to release the software to production. Then, to the extent that it is infeasible to do all that stuff, scale back to something that can be accomplished within a Sprint. Then, continuously improve (i.e. inspect and adapt) until the definition is once again everything that it would take to release the software to production.

This question has been at the forefront of several conversation I have had lately both with highly experienced Agile coaches and with new, interested parties at the client. The question has been answered many times from many different points of view, and most of us who have been around Agile for a while think we know what it is. There are a few misconceptions out there, though, and I want to clear the air. So, what is TDD, really?

Test-Driven Development is a disciplined approach to software design that provides a mechanism for programmers to create working software in tiny increments of no more than a few lines at a time. The reason that this is important is that Agile approaches to software development more or less require us to deliver working software in small increments that fit inside a two-week iteration (give or take.) In order to have working software that meets some customer need in such a short time frame we need a way to create even smaller increments of software, see them work, build a little more, see that work, etc., and improve the design as we go so that we don’t create a mess.

The way that Test-Driven Development accomplishes this is by leaning on unit testing frameworks to build a tiny test that specifies a tiny increment of behavior. We write that test before we have written the production code that will make that test pass. Then we write the code, taking care to do no more than is absolutely necessary to make our test pass. Finally, we take a step back and look for indications that our design could be improved, the main one being duplication that is created by bits of code that do or say much the same thing. When we see such duplication we refactor it using techniques and tools that allow us to change the code safely without breaking the test.

We repeat this process over and over. Each cycle takes no more than a minute or two, and each cycle builds incrementally on top of the prior ones until we have created some working feature that the customer has asked for. All the while we are seeing all of the small increments we have built work, via the tests. All the while we are seeking and exploiting small opportunities to improve and simplify the design of the code. And, in the end we have demonstrable, working software that is capable of filling some customer need. All we have to do is integrate and deploy it and we can safely move on to the next thing on our list.

That is more or less it. So, now that we have talked about what Test-Driven Development is, let’s take a moment to discuss what it is not:

Test-Driven Development is not about testing, not really. The purpose of tests is to verify the quality and functionality of software. The purpose of what we are doing is to set an expectation of what we need to do next and have a way to quantify when that expectation has been met. We get verification as a result. That is: later we can run the tests and see them pass and be assured that no regression has happened (At least in the behavior that we specified.) But, that is not the goal that we set out with. We set out with the goal of driving small increments of code that actually work. We needed a way to define what those small increments were, and the test framework gave us a harness in which to run tiny increments of code, set expectations about how the code would work, and see those expectations satisfied.

It is important to realize that Test-Driven Development is not about testing, or quality, because it is important to realize that quality is not achieved through TDD alone. Several studies have shown that software produced with TDD tends to have less bugs, but that is a side-effect of TDD and not the goal per se. In order to have quality we need to be able to verify that the software works the way the customer intends. We need a way to verify that it looks good, is usable, accomplishes work that the customer needs it to do, performs adequately, is free from obvious defects, etc. Test-Driven Development is not about these things.

Quality, in Agile, is achieved by working in small increments, by collaborating to set clear expectations up front, by using Test-Driven Development to build the software, and by performing various kinds of testing to assure that the product meets the goals we set early and continuously throughout the process. Test-Driven Development is only about how we build the software. We need other techniques to assure that the software meets our customers’ needs.