I had a great time meeting a bunch of cool Agilists at the Agile Boston meeting on this past Wednesday night.  I had a chance to present on a topic I am particularly passionate about: Agile Metrics and Diagnostics.

The pdf of the presentation can be found here:

Uncle Bob Martin says programmer certification is bad. He supposes that if you start giving out certifications to people that someone will assume those certifications mean something. They will start hiring certified people preferentially, or stop hiring non-certified people entirely. This would be bad, because certification is almost certainly a poor measure of whether you should hire someone. Uncle Bob is right.

Ron Jeffries suggests that the Scrum Alliance ought to just drop the word certification. He asserts that sending people to classes is useful because they get exposed to ideas that they may not be adequately exposed to otherwise. However, going to a class should not be called “certification,” because it is imprecise and alienates parts of the community it means to serve. Ron is right.

Tobias Mayer has complained that the new Certified Scrum Developer course is no better than the Certified Scrum Master course since both are merely a couple days of training during which a human being can only retain so much. Tobias is wrong for a number of reasons. First of all, he’s not really in a position to judge something he’s never seen or done. He is also wrong because the CSD class has the one thing that all of the official Scrum Alliance classes have lacked to date – relevance to software development.

Attendees of a CSD class have to write software. This is supposed to have something to do with Agile, and the Manifesto says we value working software. So, maybe we should write some? Tobias is right to say that this isn’t enough, but it is the very first step in the right direction… ever.

I believe that it is absolutely valuable to attend a CSM or CSD course. The CSM course ought to be called, “Intro to Scrum: Do Not Try This At Home.” The CSD course ought to be called, “Intro to Extreme Programming: No Really, Someone Might Get Hurt.” The two of them together are barely a starting point. They are a little bit better than reading a book and trying to get someone to pay you to attempt what it says (The way many of us learned, including myself.) They still aren’t enough by themselves.

To be able to do this stuff well, I think you need at least the following things:

1. A commitment to do your very best and always strive to do a bit better each time.
2. Some experience attempting these practices in a setting where mistakes don’t matter: a user group, a class, doing katas on your own or with friends or coworkers.
3. Access to people who have gone before you so that you can listen and learn from their experiences, and a willingness to do so.
4. Plenty of time for these ideas to ferment and turn into something useful.

I posted this to the LAJUG mailing list, then later to the Extreme Programming Yahoo! group. Some folks really liked it, so I’m putting it up here for posterity.

The Java Programmer’s Rules of Design are, in order of priority:

1. Should not have any tests, because I wrote it so it works.
2. Should clearly express how clever I am.
3. Good idioms should be repeated often. Better idioms oftener.
4. Should contain as many classes and methods as I might possibly need
   at some point in the future.

(Read here if you don’t get the joke.)

P.S. If you are a Java programmer and this makes you sad just substitute your least favorite of: C#, C++, or Perl. It works for them too…

Hopefully you recognize that this is a play on the term information radiator, but let me explain how/why?

I find that when starting with a new client and discussing the idea that we are going to put all their tasks and activities on a wall for everyone to see and so that it’s easy for the team to update, there is almost always push back or at least hesitation. The concerns typically are centered around the fact that “well we already have a dashboard system we use here to track project status and Sr Management uses it”. My guess is they have never seen a good story board/task board wall.

So where does the refrigerator part come into this?

Well, recently my refrigerator went on the fritz and stopped working, but I didn’t know it. I noticed that items coming out of the fridge seemed to be less cold, but I figured it was just me. So I didn’t do anything. Many hours later, the problem seemed to be worsening. Lo and behold, we had a problem and ended up losing a bunch of food in the refrigerator.  We had it repaired and all was fine, until it broke again.  This time the “breakage” was more obvious, but only because I was standing in the adjacent room (there was an electrical popping sound and a puff of smoke that came out from under the unit).  If I hadn’t been in the next room, again I wouldn’t have known something was wrong.

In the end, my wife and I decided to buy a new refrigerator instead of repairing the original.  The unit we purchased was half the price of the original unit (which was only 7 years old) but has more features and fit our needs better.  We purchased based on price, look and features, in that order.  It has a feature that I really didn’t notice/appreciate until it was in my kitchen and plugged in.  A digital display that shows the temperature of the freezer and of the refrigerator.  That’s the information radiator.  It even has an alarm built in alert us if the temp rises too high.  I could have a manual thermometer in the fridge and in the freezer to tell me the same information, but the difference is, I have to go look for it.  I would have to open the fridge, find where the thermometer got moved to and read it, just like a traditional dashboard.

With my new information refrigerator every time I am in the kitchen I know if things are working as expected simply by looking.  No digging for thermometers (or dashboards located on the web or in some program management tool somewhere).  So now if one of my kids leaves the door open or if it breaks (it better not, it’s brand new), I have a much better chance of knowing and being able to take a corrective action.

So what makes a good information radiator?

• It’s highly visible
• You don’t have to go find it, it’s just always there waiting for someone to look at it
• It is kept up to date – it’s simple enough that the team actually uses it and updates it and it reflects what the team is working on
• It can alert a team to a situation that needs corrective action – it doesn’t tell you what to do, it just displays the facts, we have to do the rest
• It’s simple

I love the information radiator part of my new fridge so much, I’m not sure I would ever want one without it.  Just like I would never want to run a project without an information radiator.  Once you see it and experience the simple power of it, you’ll know what I mean.

If you are a virtual team you have additional complexity to deal with.  BigVisible has built a virtual task board (information radiator) that is free for anyone to use, check it out at www.seenowdo.com

A few of you may know that the first ever Certified Scrum Developer course taught by Ron Jeffries and Chet Hendrickson was held the first week of May in Cleveland, OH. A couple of those may also know that I attended along with a handful of well respected coaches (George Dinwiddie, John Kern, Jeff “Cheezy” Morgan, Paul Nelson, and Dave Nicolette) and some other Agile developers from all around. It was an interesting class and I think an exciting move forward for our community.

There is a lot of confusion and opinion around the idea of developer certification, much of it justified. I think that Ron and Chet have effectively taken the high road on this. On one hand, they have bowed to its inevitability and decided to become the first to attempt to do it well in the context of Agile and Scrum. On the other hand, they have done their best to make sure that the course is uncompromising in showing what it is really like to work on an Scrum team following a disciplined engineering approach.

In this last regard I feel that they have been quite successful. The course was, in many ways, identical to many real projects that I have participated in except for the absurdly abbreviated schedule (Necessary to fit the experience into just three days.) It was even effective in presenting some of the usual problems: set up problems, communications problems, arriving at shared expectations, etc. What was interesting (and enlightening, and mildly entertaining…) was that even this highly experienced group of attendees couldn’t avoid some well known problems including ones that we coach teams every day to avoid.

Too many cooks spoil the broth, or so the old adage goes. It seems that maybe too many coaches spoil the product as well. One of the interesting and unforeseen consequences of having so many highly experienced people work together is that we didn’t appear to be as effective as we might have expected.

There are a number of likely reasons for that. For one thing, I think we all wanted to give due respect to our peers, who we knew by reputation but had never worked with before. For many of us, myself included, it would have been more usual to take a strong, principled stance on certain issues, but instead we may have tried too hard to just get along. I think there was also a tendency for the coaches to coach instead of just getting stories done. In a real project these sorts of team dynamics would work themselves out over time, but in the short time allotted we had only begun to gel as a team.

Overall, it was a great experience. I met some cool people, and had a lot of fun slinging Java code with them using TDD, pair programming, and continuous integration through and through. I hope that everyone gets the opportunity to experience this course and find out what the whole XP/”engineering practices” hoopla is all about. Hopefully, BigVisible will soon be involved in making that happen.

It has been a while since I wrote the first part of this post. Sorry for the delay.

In the first part of my post about essential complexity I said that there would be a second part where I explained the approach to managing complexity on Agile teams. However, I also gave the answer in the first part. The answer is Test-Driven Development.

The diagram below shows the usual way that complexity is managed on software projects:

When we start building the solution we have certain ideas about what we are going to need to do and how we are going to solve upcoming problems. We build those ideas into the approach to our solution, but along the way we discover new problems and assumptions. So, we build those onto the solution we started with. By the time that we approach our first release we have created something that contains a certain amount of accidental complexity.

Our solution is represented by the black arrow in the diagram. The essential complexity is represented by the gray arrow. In order to make our solution perform well and respond to change we attempt to drive the complexity down through refactoring. Unfortunately, at this point the solution already contains too much complexity to refactor easily. This is the familiar situation of “legacy code.”

TDD recommends something radically different as represented by the following diagram:

We create a solution that is naively, terribly, overly simple. We know that this solution won’t solve all of our problems, but it solves the problem that we have immediately posed to it (As represented by the tests.) In order to arrive at a solution that contains our essential complexity we continue to add more and more tests that ask further questions of our system. Then we work diligently to ensure that the system is no more complex than necessary to continue to pass these tests. We use refactoring right from the beginning to achieve this.

Working in this way we are able to drive from a system that is too simple to one that is just complex enough. The real beauty comes from the realization that we can often get considerable business value from incomplete solutions. That is to say that our customers benefit when we deliver solutions that do not manage to tackle the essential complexity of the problem posed to us but merely a useful subset of it. Further, our overly simple solution, which has value to our customers, is very lightweight and easy to change. So, we can add features to it at a blistering rate, and continue to learn and get feedback.

There is another advantage to this approach. Our system is always tested, and therefore testable. Those tests provide some assurance that the features we have implemented still work. So, we are always able to make changes safely and quickly. The only bugs that such a system can produce are the result of tests that we forgot to write because we didn’t think of them. However, it is trivially simple to add those tests, and generally quite easy to add the minimum functionality needed to make them pass.

Wait a minute… that sounds just like what we said we do when we add a new feature. In fact, it is. Discovering bugs is just discovering essential complexities that we have yet to address adequately. If we have a lot of complexity to manage when we do that then it is difficult, but if the majority of our mistakes are a result of a solution that is too naively simple then it is almost trivially easy. This is the power of TDD.

[Update: I changed the size of the images so that they don't bork the whole page flow.]

This is an important topic, and I intend to approach it un-gently. Apologies in advance…

Software Architecture, at its worst, is the art of creating solutions looking for problems. Unfortunately, it has the ability to achieve its worst if we are not diligent.

There is not, has never been, and will never be a role called “Architect” on Scrum or XP teams. Scrum says that Product Owners own problems, “the what”, and Teams own solutions to those problems, “the how”. The Whole Team needs to decide how to best solve the problems it faces with the simplest solution possible in a short time frame. Therefore, the Whole Team owns the architecture of the solution.

Recently, at a client, it was pointed out that if teams are allowed to solve problems however they see fit, and if multiple teams solve the same problem different ways, then no one will know what the best solution is. Chaos will ensue.

Interesting problem. Let me take a stab at it:

1. I don’t have any problem with chaos per se. It might even be good for you.
2. If the organization has multiple teams that are capable of solving the same problem, over and over, in simple effective ways, without learning anything new, then we may have a problem. It’s a problem I would love to have.
3. Given that we mostly don’t have this problem (Not anywhere that I have looked) perhaps we should try to foster teams that can create simple solutions, learning as they go, and then solve that problem when we get to it.

Reuse is mostly a red herring. There is a distinct lack of evidence that we can design for reuse and create a solution that is sufficiently simple to be flexible and responsive to change. Since Agile teams value the latter, it may not be in their best interest to attempt the former. Instead, what we tend to do is to build solutions that are more complex than the problem at hand requires (Remember essential complexity?) That complexity is like a weight we have to carry with us everywhere we go. It effects the nimbleness with which we adapt to change.

When we “architect” we bring in that accidental complexity in the name of things like reuse, performance, scalability, testability, security, etc. Not that those goals aren’t worthwhile, but they are things which are best seen in the context of what our users need to do rather than as ends to themselves. Generally, from the user’s perspective, it needs to work and do something useful first and foremost.

This is where I go back to the idea of essential complexity: first build a solution that is too simple to work. That is zero architecture. Then build that solution up incrementally, massaging and caring for the design along the way, until you have something that solves the problems you set out to solve. That is evolutionary architecture. Anything bigger or more upfront than that is probably creating solutions looking for problems.