Personally, I find this question one of the most loaded within the profession, “so how much (faster/cheaper/in some way better) will using Agile make my projects?”

This is certainly a legitimate question, but there’s frequently an undercurrent that indicates a very narrow perspective on what one is willing to invest in such a process. I do not mean to imply that there are not tactical performance improvements to be gained from using Agile, they are legion and well documented. Rather, if we focus purely on simply spinning the wheels faster, we will surrender a significant amount of potential value and ultimately limit the success of such an initiative.

Just the Numbers

Before diving too far into the details of these “other benefits”, let me speak to the first question of tactical improvements. Mike Cohn put out a pretty authoritative presentation on the topic of performance improvements, it is available to all and licensed under the creative commons. Let me share a few highlights from sources he found, as well as some I’ve come across:

• According to QSM’s Agile Impact Report in 2008, Agile teams are 37% faster to market, 16% more productive and, most interestingly, able to support crashing project schedules by up to 50% without seeing geometric growth in defects.

• Several different organizations conducted surveys in 2008 asking respondents to identify the benefits they have realized from using Agile. These include VersionOne’s 2008 State of Agile Report, Tech Target’s 2008 Agile Survey, and Dr. Dobb’s 2008 Agile Adoption Rate Survey. I have included a summary of the responses.
  

• The Standish Group’s infamous CHAOS Report has shown a marked improvement in project success rates since 1994, having moved from 16% (1994) to 29% (2004), and many within that project have specifically highlighted Agile as one of major sources of that improvement

Faster Isn’t Enough

I am not one to  begrudge someone for desiring to spend less money in order to build software faster and with less defects, but I can’t help but think of that old joke about the expedition in the Amazon rain forest. After several difficult days in the forest, the team begins to make progress. They are moving much faster and are covering quite a bit of ground. One of the men is curious how fast they’re going. He climbs up a tree, surveys the surroundings, and shouts back down, “we’re going in the wrong direction”. Someone shouts back, “but we’re making great time!”

Indeed, this is the key challenge, and potential benefit, that the values and practices of the Agile Manifesto seek to address. It’s not about improving our throughput by 10%, it’s about building fundamentally better software in a collaborative way that maximizes the value of our business partners based on their circumstances. Let’s talk through a few dimensions in which to consider how Agile can help us.

Financial Performance

Many people discuss the value of time to market in the abstract, but sometimes it’s good to offer a concrete example. Imagine two projects, one delivered after 1 year (let’s call this one “Traditional”) and the other incrementally beginning at month 3 (let’s call this one “Agile”), with the following assumptions:

• Both projects yield $3 million every year, once they are up & running.
• Both projects will take 1 year of project work to complete.
• Both projects run perfectly and at the same rate, meaning the product is fully built after 1 year at the exact same cost.
• Upfront costs are $100k and the team costs $1.25M a year to run (20% of that in future years to support).
• The cost of capital is 15%, charged over 6 periods every year

What we see is that the Agile project, under the exact same circumstances, will have a NPV value of nearly double the traditional project, due entirely to the value of earning cash flow early. I must give credit for this spread sheet to John Scumniotales, who first put together a similar scenario here. I have modified his assumptions slightly to make it more conservative, you can see my version here.

Other than the return, there are a few other points that bear mentioning in this model. Note that the Agile project has a significantly lower financial exposure ($900k vs $1.4M) and that it reaches that exposure earlier than the traditional project (month 8 vs. month 16). This means that the Agile project will bear less financial risk, and we will know if we are going to break even much sooner. Now, of course, this argument is predicated upon teams being able to plan a product that can launch early and incrementally. This is not a given, and brings us to the next point that early feedback helps us build a better suited product to whatever the need its trying to fit.

Product Quality

I must be careful when I say “quality” as many of us have been conditioned to think that simply means “less bugs”. As we saw earlier, this is a real benefit of using Agile, but it is not what I’m talking about now. What I’m thinking about are those projects that were “paper successes”, where it is brought in on time, on budget, and too spec, but unfortunately it doesn’t actually solve the underlying problem. These are a pernicious challenge for IT professionals. My very first big project was delivered according to the plan, but it didn’t do what the business needed. Now my manager told me that it wasn’t my fault. They should have given us better requirements. This explanation never sat well with me, and it is one of the things that has led me to embrace the high levels of collaboration and empirical feedback championed by techniques like Scrum.

Let me offer another example, in 1994 Fox won broadcast rights for the NHL. They wanted to increase viewership and surveys of potential viewers showed that new comers had a hard time keeping track of the puck. Focus groups indicated that these people would like a way to better track the puck as it moved around, and responded very favorably to the idea of visually highlighting the puck on television screens. The network was under a tight deadline for the NHL All-Star game in 1996. A team was commissioned to achieve the task with $2 million. The outcome was far from certain, even up to a week of the event. The team tried a variety of technologies including radar before settling on a series of infrared LEDs to be contained within the puck and a relay of sensors around the ring hooked up to quite a bit of processing power in order to add visual effects when the puck was in the air or traveling faster than 70 miles per hour. Ultimately the team delivered on time and it was a technical success. Unfortunately FoxTrax has moved on to hockey infamy. As one reviewer lamented, “I watched a hockey game Saturday night and an episode of ‘Star Trek’ broke out” (St. Louis Post-Dispatch, 1/22/96).

While FoxTrax remained somewhat popular with people new to the sport, die hard fans hated it. The technology was used for about two years, and then retired shortly before ABC got broadcast rights in 1998 (more details are available here). Thus, we are faced with an ambitious technical project that was delivered, but was ultimately a market failure. Here’s where we need to be careful about the fine print, when we see things like the CHAOS report show project success rates of about 30%, they are measuring based on technical measures (budget, time, scope). They do not take into account situations like this, in fact, this project would be defined as a “success”. If we look towards Agile to simply make us “go faster”, we completely surrender the opportunity to leverage our incremental and iterative approach in order to make a better product and avoid situations like this. This is what I mean when I refer to product quality.

First, let me give a good non-IT example that I find most interesting. The America’s Cup is a series of sailing races held throughout the world. It is so named as the first ship to win the race was named the “America”, as well as the fact that the USA has the longest sporting record, having won the contest for 125 years from 1857 to 1983. For those not familiar with the sport, it is an intense contest, not only in the performance of the sailors, but also in the design and construction of ships. The only team to beat the Americans twice is the team from New Zealand and their boat “Black Magic”. What’s most interesting is the way they went about building this ship. While the Americans had invested immense amounts of resources in the best computer modeling available at the time, the Kiwis put their designers in the the shipyard with the those responsible for building the boat. They settled on a design and build two ships. Next they allocated 6 months to race the two whereby they would use one ship as a control and the other as a variable to test out different modifications. This is an elegant demonstration of using iterations to design a product based on empirical results, the performance of a boat with a given design modification. Nobody knew what the boats would look like coming out of that six month window, but it certainly was impressive. When the boat first debuted in 1995, it not only won the cup 5-0, but only lost one of the 43 races entered that year (more details here).

The Challenge of Iterative Delivery

I really do enjoy this example, but it illustrates a fundamental challenge we face when trying to run Agile projects: namely, how do we iterate on a product? Jeff Patton first highlighted this dynamic (to me at least) in a post and subsequent presentation he gave about embracing uncertainty. He articulates that there are two different things we do when we break apart a product: we can break it into increments, which would be a distinct and discrete component such as an engine, power train, and brakes, and we can break things into iterations, which are revised versions of a complete whole, such as the 2009 Civic and the 2010 Civic. If we focus too much on increments, then we lose the ability to actually iterate over an entire product and make improvements. This is frequently seen in those projects where someone says “we want to get this right the first time” or people are upset they must revisit a component that was already built out. If we plan our projects around building each part once, we will invariably fail. A truly Agile project, must be planned around iterating over an entire product, because we know that we won’t get it right the first time and the sooner we get the first time done, the sooner we can get feedback to maker it better.  This is born out in numerous research such as the CHAOS report, which shows that 80% of a system’s functionality is implemented in a subsequent release and 66% of all features are rarely or never used (think Microsoft Office). Now, in reality we have to blend approaches, but the fundamental point is that the sooner we get a functional product, the sooner we can get meaningful feedback to build a better product and financial benefit for the company. I will offer one more non-technical example.

Imagine a house, if we build a house increments (the foundation, the walls, the roof, etc.) then we don’t have something we can live in until the end. We also face maximum financial exposure upfront, and must make all design decisions at that time. Whereas, if we build a home and expand on iteratively (a dormer, a garage, a new wing), we can continue to refine and enhance the house all the while having a place to live in simultaneously. Of course this is not a perfect example, as software is infinitely more malleable than wood, brick and mortar, but I hope the analogy demonstrates the concept.

Personally, I think this is one of the most exciting and interesting things about Agile projects, as we must be creative in the way we frame them in order to take advantage of iterations. My first “aha’ moment in this field was with a financial service company building a new webpage. As time went on, they became increasingly frustrated over performance. We had not surfaced it at first, but the page’s performance was a primary concern. So we set up a new feedback loop, and ran a very crude performance test in the development environment with every hourly build. It did not extrapolate perfectly to a true performance environment, but it gave us an excellent tool to engage and discuss with the customer about just how much they wanted that next feature, as well as how much it would cost them. We had total transparency into how much each new component impacted the performance of the page and this led to a website that looked remarkably different than what people originally imagined, but it met their primary concern, and was better suited to their needs than what they originally conceived.

Responsiveness to Change

By now, I hope you all agree that we won’t ever get our requirements right up front and its a fool’s quest to try to do so. But, even if we were to have a magic wand that would give us perfect requirements, the exponential nature of change makes it a moot point. Several educators put together a video titled “Did You Know?” discussing the challenges facing modern educators. It has been wildly popular and I would recommend you see the latest version here. This presentation discusses several points that merit further exploration, but let me focus on just one, the adoption of new technologies. We are now at a point where industries can change in what used to be the cycle time of a traditional project.

Ultimately, this is our last benefit to be realized from Agile practices: the ability to change. If we work in a clear manner where we’re building just enough to meet our immediate goal, we undertake good technical practices like test driven development and we have good, frequent feedback on our product, we are in an excellent position to make adjustments to the vision of our product based on the reality around us. Now this is not necessarily efficient, and it doesn’t helps us with project success metrics if we’re looking to measure things as being on budget, on time and to the original specification, but it is profoundly valuable nonetheless. In fact, this is where we see virtuous cycles. As we saw earlier, Agile projects have less financial exposure, thus there is less at stake if we need to change path, this can be profoundly valuable when we talk about the human condition and our propensity to avoid realizing a loss. Second, we discussed how Agile helps us refine the vision of a product as we go if we don’t have a certain product vision, then change becomes significantly easier.

Conclusion

As I look at the history of project management, I can’t help but see how the legacy of scientific management as defined by Frederick Taylor has been a double edged sword. On one hand, he was the pioneer of using empirical feedback. He dubbed the term “scientific management” and proposed that all tasks be designed and managed based on verifiable measures. Unfortunately he was also the person who said that there should be a distinction between those who define the work and those who execute it. Sadly, I see this legacy in so much of what we do today in IT and it is an expensive piece of mental baggage. It implies that the truly valuable people must be “managers” thereby creating a career track that encourages people to give up their technical skills. It creates a schism between those who will do the work and those who will plan it, ensuring that there will never be full buy in. And ultimately, I think the spirit of that Mr. Taylor is in the question “how much faster will Agile make me”, as it implies that we are simply running a factory line when we build software and its simply a matter of getting the line faster. Some people have been very successful with this mental model, but my experience has been that it sacrifices a significant amount of value for the team and makes for a much less enjoyable workplace.

Thanks and I’d love to know your thoughts!

I haven’t had time to create any user guides but here are a couple of things that I think would be helpful to know:

1. You will need to enable Macro’s for the sheet to work correctly
2. If you haven’t already done so, you will need to install the Analysis Toolpak Excel Add-In that comes with Excel.  From the Excel Menu, select Tools, AddIns.  Select the Analysis Toolpak and click OK.
3. After entering your Iteration Start and End Dates, click the “AutoShow Columns” button to automatically hide and show the relevant date columns
4. In the Status field, avoid using the “Fill-Down” capability to set the status of multiple fields.  Instead use copy-and-paste for the same result.  You can copy from a single status field and paste to a range of fields.
5. To have the burndown data for a specific day show up in the charts, you will need to enter an “x” in the field directly beneath the date
6. You can modify the categories by changing the list of categories in the table of “Category Burn-Down Data”
7. You can hide specific categories by removing the “x” next to the category name.
8. Send me a note for any additional help – I’ll do my best to get back to you quickly.

I’ve included two sheets: a blank tracker as well as one containing sample data.

Bigvisible Standalone Iteration Tracker v0.9.2

Bigvisible Standalone Iteration Tracker v0.9.2 – Sample Data

Enjoy
Giora Morein
gmorein@bigvisible.com

An eager consultant, she went to the stores to talk to the line managers and other employees to get a better understanding of marketplace and the company’s clients. As she began interviewing people and asking what their responsibilities were, she kept hearing one thing: reduce shrinkage, the industry term for theft. Amazingly the vast majority of retail employees seemed to believe that their number one responsibility was to ensure that potential customers in the store were not stealing; how did this happen? Apparently, each store received only one report a month, one that measured inventory losses. Now the retailer had many other metrics, but this was the only one communicated to the entire organization. There was no strategic reason why this single report was distributed, someone began sending it out years ago and no continued to happen simply because people were used to it. Yet, this seemingly innocent little communication had changed the behavior of so many of the company’s employees. Instead of seeing potential customers as people looking for help, they were perceived as potential shoplifters!

This concept is an important one to appreciate. What we measure and how we communicate has a major impact in how we frame a situation and ultimately how we act. The law of unintended consequences invariably rears its ugly head if we try to focus too much on a limited set of measures. For those of you who are wondering where I’m going with this post, as the title would imply, I think this can sometimes be the case with story points. Now story points and their complimentary project burn-down charts are not a bad thing. Quite the contrary, they are a very valuable tool. However, they are simply meant to be a measure of the relative amount of work, the rate its getting done and a projection of how long it will take to complete. Story points are not a means unto themselves and we need to be careful to understand the role of points and not let them distract us from the more important goal of business value.

No I’m not saying that all people make this mistake, but rather that we need to understand its an inherent risk whenever we measure something that we will optimize on that to the detriment of that which we don’t measure. To go back to the analogy of an addiction, points are a valuable resource, but we can go to far and use them to compensate for other problems, thereby creating an over dependence. This will also allow other issues, like an inability to measure value, to fester. Ultimately, we are worse off for this type of addictive behavior.

So, if you are operating in an organization that does not have a clear way to measure business value, but can measure points, this becomes a distinct risk. When setting up a project, we should ask ourselves how we’re going to measure progress and what will truly deliver value. Establishing metrics around these vectors will pay high dividends later on and let us focus on the true priority at hand.

The authors first introduced the topic with a simple economic study, the impact the changing price of eggs had on sales. The researchers found that consumers were more sensitive to prices increasing than decreasing. The impact of a loss – in this case due to price increases – was 2.5 times more profound than the impact of a gain (Putler, 1992) . Quite simply, people do not rationally weight losses properly and as such this leads to irrational behavior.

This concept was then applied to numerous examples such as one of KLM’s premier captains flying his 747 down a foggy runway into another plane because he was facing a flight delay and didn’t yet have tower clearance or a financial adviser explaining how many of his clients are unwilling to sell a poor performing stocks once it is below the price they bought it at, even if the most likely outcome is that the given stock will continue to drop (Brafman, 2008). Individuals fear of REALIZING a loss that has already occurred causes them to take on additional risks that are not rational. Perhaps the best, example offered was the $20 bill auction conducted by professor Bazerman at Harvard. At the beginning of each semester he offers an auction for a $20 bill starting at $1. There is a twist; the 2nd highest bidder must honor their bid, but gets nothing. As the price moves closer to $20, two bidders will become locked into a bidding war as each person hopes to outbid the other, thereby avoiding the $20 loss. Professor Bazerman has indicated that the bidding has gone as high as $204. This idea of risking more to make up a loss is not uncommon. In your own life, think of the last time you were driving somewhere and were running late. Did you drive faster or more aggressively than you would normally?

While the authors did not explicitly apply this dynamic to large projects, I can certainly see how this dysfunction has played out in projects I’ve been on. In fact, it seems the nature of a traditional waterfall project would be quite prone to this irrational loss aversion. Imagine for a moment a project broken evenly into three major toll gates for requirements/design, construction and testing. Now imagine for a moment that the first face is coming to a conclusion, but the design is not quite nailed down, or an extra week is required. Traditional project management would say that one would communicate the schedule slippage and move the date out. My own real-life experience has been that people will say, “we’ll just make up the time!”

One of my friends in QA has frequently pointed out that the nature of his job as a QA professional means that he’s “the last one to go and the first one to be cut” implying that that traditional projects generally run longer through their earlier phases, in turn placing extreme pressure on QA to make an original date. I would like to say that when I used to work on waterfall projects I was strong enough to resist this dynamic, but sadly I was not. On several occasions, the siren call of “making the date” lured me too close to the treacherous rocks that have shipwrecked many vessels.

Somewhere deep in our psyche is a fear of loss, and with the modern large-scale waterfall development projects we have put together a system for delivering projects that basically plays to this fear. Is it any wonder we see massive projects go way over budget and yet continue to go forward? Like the ambitious MBA student looking to avoid the $20 loss, there are too many examples of companies throwing good money after bad to avoid ending what has already become a disastrous project. One client I recently worked with had just concluded a massive, two and a half year project only to realize that the cost to maintain this newly developed product was so high it could never be sold at a profitable margin.

The iterative nature and value of frequently delivering production ready code certainly help to mitigate this dynamic in an Agile environment, but I would not pretend that the same pressures don’t exist. I know I have seen business executives simply assert, “we must get everything done by this date”, expecting the team to find a way. Just because one works in an iterative way, doesn’t mean we are really open to evaluating our progress and make adjustments. From a project management point of view, we can refer to the triple constraint and inquire what is the lowest priority: cost, schedule or scope? In the unfortunate case where project sponsors are not willing to entertain such a trade off, then one must move to a discussion about risk – or reduced quality – and how much the program is willing to take. The best course of action is to surface these subconcious tensions that may be playing out. While we may not want to be the bearers of bad news, it is far better to escalate an issue when people still have options than when it is too late to do anything. Also, if we accept people are inherintly adverse to realizing a loss, the team may be much more willing to walk away from a small train wreck of a project, rather than a larger one they would be even more invested in. Unfortunately my experience has shown that the most likely, and worst, possible option is to shoulder the risk oneself and subscribe to the fallacy that it is our job as project managers to manage risks like this. Of course, now the stakes become higher as we continue to bid more and more for that $20 bill.

How many of you have seen this situation? After an iteration sprint, the team sits down for a show and tell and the product owner marvels at what has been created in the iteration. This cycle continues for several iterations and then as the team nears release a number of other stakeholders begin to look at the product and express serious concerns. Or worse yet, the product launches and reveals a major miscalculation of market conditions. My own example was working with a transactional system where the team had mapped out a wonderful guided interaction with help text, supporting content and even confirmation prompts. A new user trying to undertake this transaction would have no problem at all figuring it out. Of course, the challenge was that most users weren’t new. In fact, most users were veterans who would need to execute this transaction many times in a single day. Their primary concern was not guides or a slick interface, but performance.

Now even with poor feedback, such a situation would come to a head once the product launched and produced negative reviews. In an Agile environment, at least this would have consumed less time, perhaps several months instead of years. But still, is that good enough? Agile allows for a lot of mistakes, but that doesn’t mean we should have no diligence. We do not allow poor code quality into our system arguing we can catch it in the next release. Similarly, we should not accept poor quality in the feedback we receive as we incrementally build a product. The example above is an example of a poor feedback loop. Much like a properly run retrospective or scrum, effective iteration reviews and demonstrations require skill and expertise to be effective. Simply walking through a piece of functionality on a projector in front of the team is not necessarily enough; we need to look back to our business case to properly demonstrate our progress.

In waterfall, teams face the chronic risk of working towards the wrong metric. Delivering the various SDLC artifacts in a timely fashion does not guarantee a successful project. Agile teams are ideally measured based on the results of their demonstrable product, but what if the demonstration is not a realistic portrayal of what the system needs to do in the market place. Every few weeks it is great to show flashy new interfaces or functionality, but if the system is being demonstrated for the sake of demonstration, then not only are we failing to getting valuable feedback, but we may even get poor feedback that would send the team in an unproductive direction. In the example I cited before, going through a single transaction in such detail lead to wrangling over how much content and support to show, as well as concerns over fringe cases that were making the component much more complex.

So how does one address such a situation? The immediate reaction is to launch often and early. Nothing focuses a team and gets the attention of stakeholders like an impending release. But, this is not always a real option. While we may have a product that is “potentially shippable”, what if it is part of a larger service offering that is simply not ready? We can’t get feedback from our customers if we don’t have a complete enough product for them to actually use. The business constraints may mean that a team must work for months, or maybe a year before they have a viable product that can be launched, how do we ensure valuable feedback prior to that launch? The rules of Scrum dictate that ultimately the product owner is responsible for delivering the business case, and I agree. My concern is that sometimes we as a team assume that is  the sole responsibility of the product owner and is independent of the team. In this case, people invariably focus on how they feel they will be measured: the show and tell demonstration. As the old adage goes, “be careful what you measure, as that is what you will get”.

If the team can not clearly measure and communicate business value delivered, management will seek another measure to gauge progress. Sadly, the most likely culprit is story points. I am always amazed at how quickly people, who had no prior concept of the idea of a story point – a sizing measure only relative to work done by a single team – will begin demanding that a team deliver some arbitrary number of points. Just as the entire product is the joint responsibility of the team, I would argue that demonstrating and measuring business value is the responsibility of the whole team. If we were to look at the case I mentioned before, the team could have built test cases around how quickly someone could run through 10 transactions in rapid succession or run internal betas with larger groups of stakeholders within the organization as the show and tell.

The potential value of having working product every iteration is immense and we should not waste it. As business and technology partners work closer together in Agile teams, there is shared responsibility in both directions. Business partners give up the illusion of a contract-based, “guaranteed” delivery, but in return we as a team must do everything we can to show them what the product can do – and not do – at each given iteration. This calls for creativity in precisely what we test, how we measure system performance and what we demonstrate at the end of an iteration.

Poor feedback can not only be counterproductive, it can be dangerous. Let me offer one example outside of the IT world. 1975, Pepsi started the “Pepsi Challenge”; people were given tastes of Coca-Cola and Pepsi and asked which product they preferred. Even though Coke had a much larger market share, the majority of people favored Pepsi in these tests. Coke conducted their own surveys and found similar results. For several years they developed a recipe based on taste tests and ultimately in 1985 they launched New Coke. After mixed early reviews, the market turned against New Coke so harshly that in less than 3 months they were forced to relaunch the original recipe as “Coca-Cola Classic” and within a year New Coke had a market share of approximately 3%. Experts now argue the sip test was an incorrect measurement of customer preference; people don’t drink a few sips of soda, they drink a whole bottle. Unfortunately by this flawed measure, New Coke did quite well. It won taste tests over Pepsi. Unfortunately, that is not how the market ultimately values soda (Gladwell, 2005). This story did have a happy ending for Coca-Cola, as sales of Coca-Cola Classic exceeded prior sales, but that does not change the fact that this was a huge embarassment and waste of resources for Coke.

I am not a methodologist, though I love to learn about all kinds of software development methods. They seem to cover all bases. It all makes sense – no more mysteries. Surely when following these steps, I won’t miss a thing! Seeing a detailed method laid out in a process flow diagram gives me comfort. Every artifact I can imagine is documented and approved somewhere. I can see where everything goes from idea to code to product. I will know where we are going wrong every time…The promise of complete knowledge!

So why don’t these methods result in more success?
I have become convinced that 2 major reasons (in addition to the commonly-discussed problems) they don’t result in more success are

1. there is just too much ‘method’ in SD methodology. There is too much ’stuff’ to do, and
2. by virtue of having steps and artifacts assigned to roles, people are no longer responsible for the product, they are responsible for ’stuff’ – artifacts and steps

Too much ’stuff’

All these steps, all these artifacts – all this stuff. Add all these things to the chaos of every day – last-minute issues, being split across multiple tasks and projects, multiple bosses. Who has time to understand the ‘Spirit’ of the methodology – that which actually is needed to make it work?

Methodologists can look at the RUP, at anything Summit-D based, at PMBoK, at any other heavyweight methodology, and see how many of these methods may actually be customizable…How with some thought you actually might be able to run an Agile project using them! How they may in fact be iterative, despite the linear appearance of the diagrams…

Most people (myself included) are not methodologists. They are people trying to get projects done amidst a host of challenges and distractions. There is little time to invest in understanding how a holistic SD method works, how to customize it, how to somehow iterate on one dimension while performing linear phases on another! Though these things seems simple to methodologists…they are a layer of complexity that most of us on the ground can’t even begin to devote the mental cycles to.

Accountability for Product is Lost

On a project, I am no longer responsible for the finished product. I am responsible for an artifact, during a phase. When the phase is over, and my artifact is ‘done,’ my job is done, and the next step (and eventually, the product itself) becomes someone else’s problem. Successful product development now relies on assumptions like these:

• That my artifacts got the true user requirements right
• That I consumed the last artifacts and interpreted them correctly when creating my artifacts
• That everyone consuming my artifacts interprets properly when they create their artifacts
• That the final product, the result of all of these handoffs and interpretations and translations, is what the user truly requires
• That nothing changes

There are many challenges with these assumptions. If people are not responsible for the finished product, but rather for interim artifacts, is there something lost? Is their expertise available once they hand off their artifacts to others? I believe that something is inherently lost just by virtue of changing any person’s goal from the finished product to any interim artifact.

Agile

I believe that one of the most important unwritten benefits of Agile is that everyone on the team, as a team, is focused on the goal of the end product. Artifacts can be used as tools or as consumables in the achievement of the product. Everyone has the same goal. No one can say ‘it’s not my problem, I completed my artifact.’

‘Hybrid’ Methods

Some very talented methodologists, as the pendulum swings from pure prescriptive methods to pure adaptive methods towards more prescriptive again, propose ‘hybrid’ methods. Get the benefits of both! We can ‘balance agility and discipline’ (inappropriateness of those specific words when comparing Agile and traditional methods is a separate topic) We can benefit from agility while gaining the comfort of our artifacts…panacea. Hybrid methods look great on paper. In practice, I’ve not seen better results – once you start introducing the artifacts as ends themselves – the focus away from the product itself begins. The benefits of the Agile principles quickly fade away.

Consumables, not Deliverables

I like to treat artifacts as consumables or as tools – things that can be used up during the process of creating the end product. The better the quality of these consumables – the more ‘pure’ they are, and the less unnecessary stuff they contain – the easier they will be to process. Artifacts can also be tools (many examples of how can be found throughout the Agile world on the internet) – used as needed when they help teams produce great product.

In the past two years, this client has moved from a crappy Agile implementation to doing Agile the right way to implementing a lean software development system. They have taken the concept of smaller batches, removal of waste, and continuous flow of value to heart.

A common problem in many organizations is that business often pushes projects into an IT organization faster than IT can complete projects. Some of those projects simply accumulate within the organization, become work in process, and slow down the system. To counter this problem, the business at this client ensures that they do not overfill the hopper and clog the system (IT).

It is all well and good to make sure that wasteful activities are being removed from the development process; it is another to make sure that the right projects are being worked on.

As part of the initial engagement, the Product Owners and I go through the complete wish list to determine the cost and benefit of each of the projects or enhancement requests. As an example, there were 55 line items in their 2008 Value Stream Analysis spreadsheet; most will never see the light of day as their relative ROI is poor compared to the others on the list.

Funding is asked for and approved based on the work done in this joint session. Only projects that meet a certain ROI threshold are approved. Project funding is denied if there are questions surrounding the ability to meet the targeted ROI. The benefits are evaluated 6-12 months after a project is delivered; shortfalls are analyzed in an effort to learn and to not repeat the same mistake.

The modus operandi is unlike that in most agile implementations that you see elsewhere. There are no iterations; instead, requirements in the form of stories are pulled by the team as they complete the stories that they have been working on. The Product Owners and I make sure that the backlog is always kept prioritized and that the top of the list is well thought out and defined.

The team is not collocated with the Product Owners — instead, most communication occurs via Skype and GoToMeeting. This actually works remarkably well for explaining the stories, clarifying issues, and demonstrating completed work.

Iteration planning is replaced by Skype and GoToMeeting sessions between the team and the Product Owners just prior to the start of that story. The team picks up a story or two from the top of the prioritized backlog and calls the Product Owners to discuss and ask for clarifications. The teams break up a larger story into smaller stories if necessary and begins test driven development.

Informal demos happen as the story is being worked on (and an interesting portion has been implemented) and when it is finished. There is no formal end-of-iteration demonstration. Feedback is incorporated immediately and the story re-demonstrated. Once the Product Owners are satisfied with the story, it is ready to be moved into the QA environment; promotions to Production happen every week.

The teams working for this client are significantly more productive than other comparable teams following the standard Agile way of developing in 2-week iterations. I think this increased productivity is due to the increased focus on a story or two at a time, quicker feedback and turnaround, minimization of in-iteration overhead, and continuous effort for removal of wasteful activities.

While this level of fluidity is more than what most new Agile teams can handle, they can still attempt to implement some of the practices that lead to a continuous flow of value. Story boards can provide a quick indication of and help regulate the amount of work in progress. Frequent demonstrations can speed up delivery and make the end-of-iteration demonstrations a breeze.

Employees were mandated to enter all tickets (work requests, enhancements, and bug fixes) into a requirement-tracking tool. All tickets were to track through each of the following phases.

• New: All new work requests (tickets) should be entered into the requirement-tracking tool by the business analysts or by the development teams. Tickets must be entered at least 91 days before a scheduled monthly release.
• Acknowledged: A service level agreement (SLA) mandated that the business must acknowledge tickets within 7 days of receipt of the ticket – 84 days prior to release. This 7-day period was used to triage the tickets to remove duplicates, close tickets if no action was going to be undertaken, and to prioritize the items.
• Accepted: The IS department is tasked by the business to analyze and estimate the tickets — target 63 days prior to release.
• Business Requirements: The business analysts were required to complete the business requirements 42 days prior to release.
• Committed: IS would commit to the tickets, 28 days prior to release, that it thought it could complete — create functional specifications, construct, unit test, and build within the release dates.
• IN/OUT List: Final date to decide whether tickets are going to be included in this release or not.
• Constructed: IS completes coding, unit testing, and delivering final build 14 days prior to release.
• Resolved: Business analysts complete all testing 7 days prior to release.
• Closed: Code merged into the production environment (“going live”) on implementation day; Business signs-off on the release within 5 days of implementation.

The goal of any development system is to deliver customer-defined value, effectively, and with a minimum of waste. Systems that deliver value more efficiently are preferable to those that deliver value in a relatively expensive manner. One of the greatest contributors to inefficiency, poor quality, and high costs is waste (activities that do not add value to the end user).

While the “glide plane” concept sounds good in theory, it suffers from a few major drawbacks:

1. The glide plane does little to eliminate wasteful activities that do not add value to the end user. At Freesurance, over the 96 day (2304 hour) glide plane cycle, value is added during only 10 business days (80 business hours) during construction. This evaluates to a best case scenario of approximately 3.5% of the available time being used for value add activities. In actuality, overall productivity is further degraded by excessive rework and multi-tasking. Ouch!
2. The glide plane ultimately leads to excess work in progress — too many requirements are being investigated, sized, designed, and implemented at the same time. Time is wasted on investigating requirements that are not committed to being delivered and on backing out, at the last possible moment, implemented requirements which no longer hold true (scrap and rework).
3. The overlapping nature of the glide plane deliverables ensures that team members are not working solely on the current iteration; their focus is diluted by having to wrap up the previous release and to begin actively planning for the next release.
4. Multiple streams of work means that any single stream disruption can disrupt all streams. Critical bugs introduced in an implemented release have to be fixed, tested, and redeployed into production, therby, cutting the time available to work on the current iteration.

We have fostered corporate environments in which we place greater value on conformance to a plan – or commitment to a goal – than we do on maximizing delivery.  Most organizations I encounter provide little or no real incentives for teams or individuals to pursue ambitious, often unachievable goals.  On the contrary, the way management and customers react to a team potentially missing stated objectives forces teams to set the bar as low as possible. 

In order to limit the chance of failure, we find ourselves constantly answering the question: “what is the least amount we can deliver to keep management satisfied?”  This hardly seems like a winning business objective.  Failure today is defined by how much reality deviates from the states plan.  Because we can do little to alter reality, to minimize the risk of failure, people become be hyper-conservative.  This is what happens time-and-again on project teams.  As key dates approach and scrutiny increases, people seek to minimize their risk of failure – as it is defined today. 

This principle is pervasive in most organizations.  Hardly a week goes by without someone asking:  “What has changed that is causing us to change the plan?” or “What has caused us to deliver less than we ‘committed’ to?”  In practical terms the answer does not matter. 

I am sure there is a sports analogy that escapes me that could be inserted here, but the bottom line is that we need to create a mindset where failure no longer means not ‘completing’ everything – but rather not ‘attempting’ to.

Customers often bring up the word ‘commitment’.  I hear this word thrown around in almost every organization I encounter.  It frequently is used in the context of “Why have you failed to deliver what you committed to?”  It’s a very handy word to beat someone over the head with.  The problem is, it implies that a team somehow had the option of whether or not to meet their commitments.  That somehow a team chooses to ‘under-deliver’.  Obviously this is not the case.  The truth is we cannot ask a team to commit to a body of work — we can only ask them to commit to doing their best to achieve it.  When we ask more of them – they will simply commit to as little as possible.

I remember as children it was encouraged to have lofty goals.  I remember a teacher once advising me to set goals that might seem out of reach – and then strive to achieve them.  That was often pointed out to me as a way to “maximize my potential.”  As adults, perhaps we should follow that same advice.

But on a few different teams I have worked with, I noticed some interesting things happen which I can only attribute to human-nature.

At different times on just about every project there are the tasks and activity that nobody likes to do. You know what these are: the thankless stuff that has to get done that people avoid like the plague. Script-writing, documenting, tedious refactoring, javascript debugging… — the list goes on and on. Given a choice, most times individuals will opt for more interesting, favorable activity. This causes a challenge for the self-organizing team. It it true that on small teams packed exclusively with top performers it is more likely that team members will self-organize to make the most responsible decisions — ie: doing the “right” thing vs. doing the “easy” thing. However on most teams I think the conflict between responsibility and convenience is ever-prevalent. A team needs a leader to ensure that it does the right thing. This requires the ability to effectively communicate, motivate and negotiate. I have witnessed the effects of teams without a leader and it typically results in deterioration in performance even under the best conditions.

Some Agile purists may chastise me and suggest that each Agile team member is a leader. I simply don’t buy it. I have seen teams and organizations that try to “lead by committee”; in my experience it always fails.  On the majority of teams, self-organization and consensus typically result in solutions to problems that represent the lowest common denominator — the option most palatable to the group. Such is the result of any group negotiation where the majority must buy-in and agree. But this lowest level of acceptability typically does not represent the “right” option, or the “best” option. Throw in the effects of “group-think” and the situations will deteriorate still further.  The most successful teams are those with servant leaders who exude a strong sense of leadership. It is often difficult to quantify, but you know good leadership when you see it.  Every team I have observed – Agile or otherwise – has failed or succeeded in large part based on the effectiveness of the team leader.

The team leader can be explicit or implied but his or her importance should not be minimized or diminished under the banner of self-organization. Nor should we downplay the importance of leadership when formulating or populating team.  It is through strong, effective leadership that we can create the most empowered and effective team.