In his 1986 paper, No Silver Bullet — Essence and Accidents of Software Engineering, Fred Brooks describes the difference between essential and accidental complexity in software systems. Software is complex because what the users need to do is complex. Such complexity cannot be eliminated, but it can be managed, and most of what we do in software development is about managing that complexity.

The goal of Agile Software Development is to deliver a continuous stream of valuable, working software to customers. The challenge in doing that is to find bugs as soon as we create them, fix them, and deliver software that is defect free. This is important, because defects found by customers create a backflow of information that disrupts our ability to deliver value. To some extent this backflow is essential, but by finding and eliminating defects at the source we can keep the signal-to-noise ratio high and get our customers to give us more information about what they need and less information about mistakes we made.

So how do we, simultaneously:

1. Embrace essential complexity that comes from the user domain?
2. Eliminate accidental complexity – in all of its forms – as waste?
3. Identify and eliminate defects at their source?

The Agile approach to managing essential complexity is very close to the approach that Fred Brooks advocated so long ago: Grow the system incrementally. Hide complexity in a modular system with various layers of abstraction – individual ideas are expressed where relevant and viewed as abstractions from the perspective of surrounding modules. All software paradigms, object-oriented, procedural, functional, aspect-oriented, etc. are methods of organizing code into such abstractions. The trick is that we can’t accurately predict where our system will become complex. So, we have to keep our solutions as simple as possible, and hide complexity only where we find it in the course of trying to describe some real world user need.

The implications of this cut across everything we do:

• Given a user story we should create the simplest solution that could possibly work. This is literally the same thing as saying, eschew accidental complexity.
• We refactor to simplify the design of existing code. This activity, which is an integral part of TDD, is specifically designed to remove accidental complexity as it occurs, particularly in the boundaries between stories and surrounding essential complexity as it is discovered.
• We move forward deliberately, in increments, with plenty of slack. We do this because complexity is hard. We need to take the time to look for it and deal with it.
• We test and refactor mercilessly. Complex things are hard to test. Just like insects are attracted to bright lights, software bugs are attracted to complexity (Essential or accidental.) Like a porch light on a Summer evening, if it’s complex and it isn’t tested I promise you it has bugs.

In Part Two I will address the issue of designing code and tests in a way that:  hides essential complexity, avoids accidental complexity, and exposes most defects at the source.