“not after-the-fact rationalization or a good story designed to hide failure. It is a rigorous method for demonstrating progress when one is embedded in the soil of extreme uncertainty in which startups grow. Validated learning is the process of demonstrating empirically that a team has discovered valuable truths about a startup’s present and future business prospects.”
Ries, Eric (2011-09-13). The Lean Startup (p. 38). Random House, Inc.. Kindle Edition.

At first blush it seems this concept is just made to be utilized by teams working in an iterative manner. They can define sprints, validate learning, and adjust course. The challenge is that validated learning is more than just conjecture or forecasts, this means we must align the product of sprints with empirical, measurable goals. Enter the sprint goal.

More Than Just a List of Stories

When I first started using Scrum, the team I was on had the concept of a sprint theme or goal. Unfortunately for me, at that time I didn’t really appreciate the power of this concept. In fact, we just used it as a summary or aggregation of what was being worked on for the upcoming sprint. “Hey it looks like we’re doing a lot on the search component this sprint, I guess the theme for this sprint is to enhance search”.

This is a common concept I still see many teams do, and it is not without merit. Product backlogs can quickly include hundreds of stories for a given release, and productive teams may complete so many even in an iteration it can be easy to lose the forest for the trees. The simple construct of themes or goals by iteration help people outside – as well as within – the team get a general idea of what’s going on and when. It is a useful tool of abstraction that can help us organize our work. However, if we simply use the theme or goal as a summary of what we already laid out, we’re massive amounts of value on the table. Let’s look at an example.

The product owner of a team I was working with was quite worried. They were building a new reporting system and progress was very slow. The technology they were using was incredibly old, and they were constrained to it for the time being. He wanted to make a release in about six months time, but was unsure if they could do it. After a couple sprints, he said he wanted a realistic approximation of the team’s ability to build one self-serve report from beginning to end. Earlier iterations had been less successful as the team was doing research for other capabilities within the system, there were some technical issues to work through, hardware to procure and some other distractions of that nature.

The product owner cut through this ambiguity by simply stating that for one 2-week sprint, he wanted the team entirely focused on one report. The goal was to see if the team could build one report from beginning to end in the sprint; everything else could wait. His motivation for doing this was to see if the plan of completing a series of reports within the timeframe outlined was even realistic. There were some 20 reports and if the team couldn’t get to a pace of nearly 2 reports an iteration, they had no hope of completing the project with this technology platform. The sprint commenced, the team worked heroically, but by the end, they were unable to complete the report to the product owner’s satisfaction. They had validated their fears that the old platform was too brittle, slow and encumbered by patches and technical debt to allow them to work at the pace desired to make the goal. Nobody could justify the failure or couch it by saying the were working on other things, they put all that to side for validate the fastest pace they could achieve. This is not to denigrate the team, or even the outcome. They learned a valuable lesson and abandoned the release, rather than throw good money after bad.

Now here’s where things get interesting. The product owner was a creative person, and money was already budgeted for this team to work on this project. While they couldn’t build the self-serve reporting system he wanted, their mandate was really one around cost savings. The company had presumed the self-serve system would deliver those cost savings, but now it had become clear that the cost of building such a system would not be justified by the value of the savings. Fortuitously, in an earlier sprint the lead architect had played around with a tool that automatically compiled reports form a series of spreadsheets. This may not sound like much, but the business unit the team was trying to support spent an inordinate amount of time compiling large reports for major customers. The process was tedious, error prone, and sufficiently expensive that they could only product regular reports for large clients. This tool the team had tinkered with briefly had the potential to automate that compilation process. I knew this team was destined for great things when the product owner asked me, “what’s to prevent me from pursing this other avenue?”. I replied that he had justified the investment and he was ultimately accountable, so if it delivered value, then nothing was standing in his way. He was quite excited and set a new goal for the next sprint.

The team believed the tool could be used to automate the compilation of reports that the business produced, but this was still conjecture. The goal for their next sprint was to validate that the tool they had used could indeed produce these reports. Based on what they learned, they would decide if this solution was worth pursuing or not. Based on this goal, you can imagine that sprint planning looked a little different. The product owner was less concerned with bugs and documentation. Several components from their “definition of done” were relaxed as the real goal was to see if the technology would work and perform in a reasonable time. With that in mind, the team chose the most important parts of the system to deliver in order to validate that learning. They constructed a proof of concept system that would be limited in configuration, error handling, and functionality, but it would consume actual spreadsheets from the business and produce a report in the same format as those used by the business. At the end of the two weeks, they were able to demonstrate that, while the system was a little cumbersome to use, it could produce a report from the data the business would use and it could do it quite quickly. The team pivoted to build a production ready version of this system.

They rebuilt their product backlog to deliver a system that would compile reports from spreadsheets and product high quality polished reports according to some predesigned formats. They were able to complete the project according the original timeline and the time savings it delivered met the initial goals of the project. To make matters even better, the cost of producing these expensive reports was lowered sufficiently that the business could offer them to all clients without changing their staffing model. The project was a great success, but I can’t help but wonder what would have happened if not for the critical activities of those two sprints.

Two Goals, Two Types of Learning

Looking at this example, we see two types of learning. In the first instance, they set a goal to test their pace. Based on their demonstrated pace, they made a decisions about their ability to deliver a project according to a projected schedule and budget. They effectively “failed fast” because certain assumptions were invalidated, namely that they could build self-service functionality at a rapid pace in the old technology platform. In the second case, we see learning about the product. They decided to try a new approach, for which limited business analysis or technical research had been done. Instead, they set a goal for their sprint of building enough to answer the necessary business – can they produce the same report? – and technical – will it be fast & automated? – challenges. In both cases, we see that the goal was instrumental in planning and scoping the sprint.

In the first case, the team decided intentionally to not work on anything other than the report. This was because they were still doing some general set up and other supporting activities. They presumed that eventually that work would go away, so if the goal was to see how fast they could build a report without distractions, they deferred those things to run a more realistic test. They even reduced due diligence and preparation of work for successive sprints so they could completely focus. They had strict guidelines about overtime; if their baseline incorporated people working extra hours, that would become expected going forward. In the case of the second sprint, the goal was focused on a specific deliverable to answer questions of fit for purpose. They were more interested in a technical proof of concept that addressed a couple key areas. The Scrum concept of “potentially shippable” code was discarded, they weren’t trying to get something to market yet, they were trying to get something to answer questions, and fast. If the technology worked, the product owner had a lot of selling and justification to do in order to make sure his own sponsors were comfortable with the change in course.

From this perspective, we see that a good iteration goal can help in a couple ways. If provides laser like focus on meeting real goals, this helps focus the team on what’s really important, and helps protect against the mindset that we need to build everything so it doesn’t really matter how we get there. Additionally, focused goals help a team confront risk, validate assumptions, and expedite learning. Properly used, a sprint goal is a very powerful tool.

The High Hurdle of Validated Learning

I must confess this entire experience occurred for me long before the concept of the Lean Startup was articulated as it has been today. I didn’t coach this team with the idea of validated learning up front, but let’s take a closer look to see how it holds up. Validated learning has some rigor behind it to protect a person, team, or company from simply justifying failures as important learnings. Rather, it requires validation that can be demonstrated empirically, which in the case of a start up means customers. Ries tells some very compelling stories of start ups testing assumptions with key metrics that can prove or disprove them. For example, in one case they were redesigning the website to try different marketing messages. They literally ran two simultaneous sites with different copy and messaging. Potential customers were randomly directed to one or the other. All the while, they were collecting conversion metrics for both sites to demonstrate empirically if one was working better than the other (Ries, p. 51). As a consultant who works mostly with large organizations in established markets, this introduces some challenges for this concept. We are still in an uncertain area, but many of these organizations sell to businesses are part of long term relationships producing far fewer data sets. Or in other cases, like some of the self serve tools we discussed, the capability is offered internally or to an existing customer. How do we validate our learning in these situations? Let’s take a look at some assumptions and key metrics.

In this example, there were two key assumptions that were tested:

1. The team can build at a rate to justify our current cost projections (6 months) – this was ultimately tested by looking at the team’s velocity, as well as a focused sprint to see if they could build one report. In both cases, the metrics showed this was an invalid assumption, which would call into question the ROI calculations for the project.
2. The team had a tool that could automate the compilation of reports that were currently put together manually – this was tested by a proof of concept version to create a PDF report in the same general format by pulling data from the spreadsheets and extracts the business already used. The team specifically measured the number of touches to run the process, the run time, and the ability to format the output. Ultimately, they were able to demonstrate a working version after their first sprint exploring this goal.

However, as I look at this situation, I see several leaps of faith and assumptions that were never validated in advance

• Providing a self-serve system to customers will reduce calls for ad-hoc reports: this was the overall business case for the original project. In the end, it was moot, as the team demonstrated they were not able to build the product at a rate to justify further investment. However, I can’t help but wonder what would have happened if they were able to go faster. Would customers have used these reports? Would we only have found out after the project was over?
• Providing an automated tool for compiling reports will save time for members of the business unit: the water is muddier on this one. While the team did work with business partners and talk about if this would be useful. They never produced a definitive measure to empirically demonstrate that it would be valuable. As it turns out, the team was able to finish this deliverable in under three months, so the risk was tacitly accepted. Looking back, I do wonder if there was a way we could have more deliberately validated this assumption.
• Smaller clients want these reports as well – this was an accidental discovery, and no one ever made an assumption about it until the functionality was available and some members of the business unit started offering it to small clients. In the case of this situation it was probably also moot, as the system only took 3 months to build. Had it been longer, the team would have had to devise a simple test, perhaps offering the report to any client who formally requests it. If nobody even asks for it, then they may not see value. Or, manually create the report for a couple test clients and see if they find is useful and want to continue to receive it.

It may be find that we didn’t validate those last set of assumptions. In one case, it was rendered moot by the invalidation of another assumption, and in the other two, the exposure was only 3 months, which the team deemed to be a manageable risk. The concept here would be to identify the assumptions and then methodically figure out ways to validate the most important, riskiest ones as quickly as possible. Considered another way, we could consider this approach to be the scientific method applied to projects. We make a hypothesis, figure out how we can test it, and then we test it. Thus, our confidence in validated learning would be related to our ability to identify valid measures that would prove or invalid the assumptions we have made in our project. While the lean start up model is focused largely on using this approach for product discovery, it could apply equally well to questions and assumptions around execution like the test the team ran to test their rate of delivery.

Getting Back to Sprint Goals

It’s probably worth reflecting on the idea that, while user stories have become the medium of work in most agile projects, they are not intrinsic to Scrum. I have not posed these questions to the creators of Scrum, but I would suspect when they first thought about planning sprints, they imagined something more like the model of validated learning than the model of taking a bunch of user stories or use cases and seeing how many would fit into the next cycle. So, with this in mind, how would we apply this concept of validated learning to agile projects and sprint goals?

Validated learning requires methodically and empirically testing each assumption underlying your project as quickly as possible. The thinking going that the sooner you can validate your assumptions, the sooner you know you are doing the right thing. Any work beyond the bare minimum needed to validate an assumption is potentially waste, because if the assumption is invalidated, then chances are that other piece of work won’t be needed either. If we look at the example of the team building the self-service reporting application, once they realized their pace was so slow they would not make the time and ROI expectations, all the work done to build out that system was discarded, thus any more than the minimum to realize it wasn’t viable was excessive effort. In the traditional agile project, we see a product owner prioritizing based on their perspective of business value. In this model, we see the product owner identifying the key assumptions and then prioritizing which ones should be answered first. Then, the team would set goals for sprints based on the bare minimum to validate a given assumption. The act of defining a sprint would be driven based on the question, “what’s the fastest way we can validate this assumption?”

Let’s take an example. Imagine we’re building an online financial research website to take on Yahoo finance. Our website is going to be better, because we’re going to incorporate the ability to relate the research to your stock portfolio. The traditional agile approach would build a thin line through the system and figure out our bare minimum to launch. We would need quoting, charts, historical information, you name it. We could outline and prioritize the stories to build our vanilla financial service website as the “bare minimum” is more important. In this model of validated learning we would want to home in on that key assumption, that people will use the new website based on the integration of stock information compared to your portfolio. That is a huge assumption and its not clear our bare minimum to launch system will even test that. Perhaps the better way would be to start conjecturing some ways to test that hypothesis with empirical data. Perhaps a widget that would be embedded in a financial broker’s webpage. Perhaps we just build functionality around a watch list, because we presume that a watch list would be used the same way someone’s portfolio would, although that in itself is another assumption we’ll have to test later. Thus, the identification and resolution of assumptions becomes the prioritizing and organizing vehicle behind the entire backlog and each individual sprint. In fact, in cases where a company may exist in an established market, preventing them from launching the full system until it meets a certain threshold of capability, may make the case for validated learning even more. Teams may start by prioritizing based on assumptions, and then at some point pivot and then focus on prioritizing for delivery on a validated strategy. Indeed, if we look at some of the ideas from other thought leaders in the agile field, several people seem to be converging on this:

• Alistair Cockburn discusses prioritizing a project first on learning, and then on business value
• Mike Cohn discusses measuring and burning down a project’s risk profile

The Art of Assumptions

Unfortunately it seems we have opened one door only to find more. Looking back at some of the projects I’ve worked on, I can see huge merit in this approach of building a backlog and iterations around validating assumptions. However, now it begs the question how do you identify and prioritize assumptions. If we identify assumptions, what happens if we can’t come up with a plan to test one of them within the time box of our iteration? Much like good teams need to learn how to slice stories and functionality, good teams will need to learn how to slice assumptions in to progressively smaller and smaller tests so that they can maximize learning. The fact that a team can’t demonstrably test and validate an assumption may also cause some question about the overall capabilities of the team. One of my favorite non-technical examples is the Kiwi boat “Black Magic”, which won two Americas Cups based on a model of two boat design. The ship builders and designers used two identical boats, one as a control and one as a variable. They would make a single change to one ship and then race the boats. If the boat with the change won, it was incorporated into the control design and they repeated the task. This went on for six months, and the team achieved a velocity of validating over a six physical design changes on a fair weather day (Michael Schrage, Serious Play, p. 182). In order to achieve that turn around they had to change the way they built boats. They brought the ship designers into the shipyard to streamline communication and also had to bear the up front cost of building two identical boats. Likewise, you may find that your team requires investments and changes to the way it operates in order to rapidly validate learning.

These questions will have to wait until another day. For now, even if we are not yet able to articulate all of our assumptions with ways that they can be tested, we can begin down that road by identifying clear business or execution goals for a sprint. Just this simple step would give us a clear validation, where we could ask, “are the stories we’ve identified the bare minimum to achieve that goal?”. It may also increase our feedback, as soon someone will start asking how you’ll even know that goal has been met. At that point, you’ll be well on your way to getting validated learning.

New research leads me to believe that we may be collectively suffering from ego depletion.

Ego depletion is the idea that self-control or willpower is an exhaustible resource that can be used up. Interestingly enough, sugar (or glucose) intake helps us prolong our ability to make decision after decision throughout the day.

Initially it sounds far fetched, until you think about all of the decisions you make throughout a work day and how they correlate with your sugar intake.

Consider the number of decisions you had to make in order to get to work this morning. Now once you’ve sat down and booted up your machine, imagine how many decisions you make before you start to even code. After you’ve started coding (or writing your tests if practicing TDD) imagine how many decisions you continue make in the span of just 1 minute.

If you do the math you begin to realize that you make a staggering amount of decisions throughout the course of just one work day. Many of these decisions are under pressure with serious implications.

In addition to ego depletion, research has found that these decisions can be broken down into pre-decision and post-decision processes.

A prolonged period of pre-decision is not ideal for a team that thrives on quick feedback loops.

I believe we can use this new found research to help our teams be in situations where they can make the best decision possible.

Daily Standups - Urge agile teams to have the daily standup in the morning if possible. It is our daily plan and we need our team focused as we make decisions on what we are about to do.

Retrospectives - Bring candy or snacks into the retrospective. Team members are more likely to forget their manners when suffering from ego depletion. It isn’t just people, it even happens with man’s best friend too.

Iteration Demos - Schedule these early and bring muffins, donuts or pastries along with some form of juice. If the Product Owner is accepting the work, he or she needs to have the mental fuel to make the tough decisions.

Feedback Loops - How long does it take to compile and run tests locally? How long does it take to deploy a build to test or production? How long does it take to get an answer from the business on feature question? All of these affect pre-decision time spans and deplete willpower.

I believe if we can align our agile and scrum coaching techniques with these findings, the result will be a team that is in an environment where they can repeatedly make good decisions.

Archetypes of Product Owners

I don’t mean to imply that every product owner I’ve worked with could fall into one of a series of buckets, nor that alignment with a single archetype would imply someone can not behave according to another. In fact, as I look at the people I have had the privilege to coach and work with, I see the archetypes representing a couple progressions individuals move through towards becoming truly effective agile product owners. These categories capture the essence of different roles I have seen people play. Some are good, some are bad, and some are somewhere in between. The most hopeful thing is that I have seen people successfully progress from one to another and to another as they learned and grew in their role. Thus, these don’t imply a fixed classification, but rather a step in a journey.

The Stenographer

This is a place where many people find themselves – including yours truly – at some point early in their career as analysts. Whether they are an analyst or a product owner, they are simply expected to ask the customer what they want, and to write it down. Unfortunately, my experience was that my customers were notorious for asking for a solution they thought would solve their problems without actually articulating what they were really trying to do. Indeed, my own journey beyond this archetype really began when I was able to eliminate a six month development project by talking through with our business partner in response to a request they submitted for some new functionality. Low and behold, we were able to offer that capability to them with the current system by using a few tricks!

As a product owner, the stenographer finds themselves – by choice or organizational imposition – in a role where they are unable to influence and not expected to add much expertise or knowledge to the problem domain. Product owners in this role may feel more like order takers from higher up executives who are looking to use them as interlocutors to relay requirements to the development team. Unfortunately, this is a self-defeating position for a product owner. They are unable to articulate a clear direction for the team to execute against, because they must continually go back to get questions answered.

Ultimately, product owners must outgrow this limited model and become masters of their own destiny if they are to help guide their teams in a coherent direction. In fact, many organizations embarking on an Agile transformation are open enough to change that this provide an opportunity for an un-empowered analyst to grab the opportunity and find a voice nobody knew they had. They begin to show leadership and vision, at least enough for the team they work with to execute effectively.

The Decider

Offering confident message and vision forward, we see product owners in the decider archetype offering a clarion call to their team about what must be done, what order and to what end. The key benefit they offer the team is decisiveness and a vision to execute against. No longer must the team troll through enormous document to glean a possible answer to a question. No more must they wait weeks to float questions in order to get simple answers. The decider short circuits these anti-patterns and provides consistent and timely information to the team so that it can continue to do what it does best: deliver software.

This is not to say that the decider has no product expertise, as some are quite experienced in their domains, but rather that their primary locus of control is in the stated role of product owner being the “single wring-able neck” who owns the product backlog for a team. They cut through the ambiguity and uncertainty so that the team doesn’t have to. At their best, they are able to set both “big hairy goals” for the project and tactical goals for the team. One precocious product owner I worked with really took to this concept. I recall working through a story map with her when we were identifying their first release. Working with some key people on her team, they decided that the priority for the first release was to confront product and technical risk. Once she clarified that goal, she quickly move through the backlog and de-prioritized anything not meeting that criteria. I recall her looking at each piece of work and simply stating things like, “well that doesn’t move us any closer to our goal, so let’s put it off until later”.

She provided a consistency that allowed the team to focus on a goal and not get bogged down in changing scope or a morass of requirements for the project that were not terribly important but threatened to distract the team.

Unfortunately, this archetype is not without it’s downside, and I would describe this as a transitionary place for product owners. While they cut through ambiguity and provide clarity of purpose, this is not always informed by deep product knowledge. Product owners working in this model run the risk of sending their teams off a cliff. Fortunately, the nature of Agile projects is designed to accommodate this, by allowing for false starts and early failure. The impact of a lost two week iteration is usually not profound for a large project that is being properly managed. This is also the opportunity facing these product owners, namely to learn more about the domain. The iterative nature of Agile projects provides ample opportunity for individuals to test a hypothesis, experiment, and learn. This opportunity is not lost on good product owners, and we frequently see them become veritable authorities within their product domains moving them into the next archetype, the genius.

The Genius

Product owners in this archetype are generally coming from one of two paths. Either they have been playing the role of product owner for some time and have used that opportunity to accumulate experience, or they are credible subject matter experts within their domain already and have now been put into the role of product owner to lead an Agile team. These individuals frequently retain the confidence and decisiveness of the decider archetype, but they can back up that clarity of purpose with a significant amount of expertise. A general difference I see between geniuses and deciders is that deciders are generally more focused on the tactical decisions. They lay out the next release or prioritize the next set of features. Geniuses will focus more energy and time articulating the big picture and crafting a bold and ambitious vision for the product overall.

This depth of personal knowledge about a product domain can be incredibly valuable to a team. It allows the product owner to better educate the team about the nature of the domain, answer difficult questions, and in general set context very well. This in turn allows the team to be more responsive and creative in their own solutions. Unfortunately, this expertise comes with a couple of risks. While a product owner with this level of expertise can be quite effective, if they do not share their understanding and help to explain their vision clearly to those around them, their expertise can effectively be wasted. Indeed, it may be quite a difficult situation for a product owner in this position, as the “curse of expertise” can even undermine their ability to communicate effectively. A critical success factor for product owners in this model is to be able to clearly articulate complex ideas. Even if they do share their expertise, they are still prone to the risks that face any expert in that their own past experiences may actually become liabilities. As the economist Kenneth Boulding once famously said, “Nothing Fails Like Success.”

Namely, patterns and approaches that have served us well in the past will invariably be used again in the future, until they fail. If an expert has used a limited set of techniques or perspectives to great success, they may be unable to let them go and move on to new ideas and concepts. Similarly, the genius archetype must avoid leaning on their own expertise too much and not exploiting the empirical feedback loop available to them inherent in any iterative and incremental project.

The other major trap that I have seen product owners in this role fall into is what Fred Brooks dubbed the “second system effect“, where they have the opportunity to build a newer version of a system they are already familiar with and become seduced by progressively more and more advanced capabilities until they have conceived of a system that is unnecessarily complex and expensive. This risk can even be amplified within an Agile project, as there are less constraints upon the scope that a product owner can request. The best way I have seen to guard against this type of dynamic is by laser like focus on tactical goals when prioritizing work for each iteration. Ironically, these are the skills that the decider archetype most embodies. Thus, some people may begin their journey as a budding product owner who is able to cut through the fog of complexity, but then later succumb to it as they learn more and more about the very system or product they are guiding.

Ultimately, the genius archetype faces the challenge of growing by increasing feedback channels available to them. The specifics for these sources are numerous and vary based on the context of the project.  It may be reaching out to less vocal stakeholders, like support or operations, in order to broaden perspectives available to the team. It could be building stronger relationships  with product development and finance in order to keep things like scope and budget within agreed upon constraints. It may also take the form of continuing to exploit iterative work in order to get empirical feedback and conduct testing and validation of concepts with the project team. Regardless of the details, experts need to ensure that they never allow their perspective become hermetically sealed within the world of their own personal experience. Having learned a significant amount about their problem domain, their journey is to continue deepening that knowledge as well as expanding the horizons into adjacent areas.

The Diplomat

The first two archetypes discussed rest upon one major assertion: that the product owner truly has the authority to articulate and prioritize work. If you read the Scrum Guide or just about any text book on Agile Software development, you will see that this is a requirement for an effective product owner. Unfortunately, I have seen too many projects where this is simply not the reality. In some cases, organizational policy, where the development team and their customer proxy are simply not empowered to make product decisions. More than anything else, this explains how people find themselves within the role of stenographer.

We explored the concept of product owners being given the authority to make decisions, as well as in some cases them simply grabbing it. But there is another path forward where product owners exert influence over their product with limited authority. This is the archetype I call the diplomat, where savvy product owners without formal authority are able to clearly and strategically work in environments to establish clarity of purpose and get questions answered so that the team can work effectively.

While the diplomat’s ownership of the product backlog may be limited, when played effectively, they can confront problems by bringing the various stakeholders together and getting them to work through their own differences. In fact, establishing a consistent vision across numerous project stakeholders is probably one of the most effective things that a product owner can do in many organizations. I have seen numerous companies where their software development process has become so fragmented and slow that no clear vision or priority exists between the business and technology partners; in some cases a consistent vision doesn’t even exist within the technical or business silos! In these situations, a thoughtful product owner can deliver immense value to their organization by bringing the different stakeholders to a table and helping them see how their inability to resolve differing priorities undermines the success of the very projects that they all depend upon.

At their best, diplomats are able to use organizational jujitsu so that people with various conflicting priorities are able to come fact to face with each other, discuss their interests, and make a decision in the best interest of the project, program and company. This is obviously much easier said than done, and I could probably write a series of blog posts discussing different techniques available to product owners in this archetype. I have seen product owners use very simple techniques to great effect, like insisting that all stakeholders agree on what to work on next, anything the group can’t agree on, doesn’t get done. This can provide powerful motivation for people to interact and work together.

While the diplomat can be quite effective and valuable in organizations where there are numerous stakeholders who wield immense influence and aren’t always in alignment, there are some limits to what someone operating in this model can achieve. Their contribution is basically limited by the ideas brought into the common pool of discussion between stakeholders. Much like the decider, they are not so much adding expertise, rather than simply cutting through conflict and indecision. In order to grow, diplomats must grow their own expertise and offer more value than simply bringing parties to the table and helping mediate differences.

The Catalyst

Moving beyond simple positional negotiations, experienced diplomat product owners move into another category I call the catalyst. In this archetype, they have accumulated considerable experience in their own right both in the problem domain, as well as in negotiation, communication and collaboration. Superficially, it may appear that they are operating in a similar model to the diplomat, but there are two key differences. They are strategic in who they engage and they are able to engage people in a way that allows new solutions to emerge.

As the name implies, catalysts play a critical role in helping get elements already within the system to react in ways that release energy or cause other valuable changes. In the case of Agile software development, this may be bringing to seemingly irreconcilable groups together to craft a shared vision allowing them to see how their interests can positively intersect. It may be pursuing other stakeholders in order to better balance the vision for a product. For example, one product owner I worked with brought several of the people from the business unit he represented to meet with the team periodically. They would show the team the type of work they did and how the project was going to benefit them. Incidentally, in one of these discussions, the lead architect happened to key into one pain point they mentioned about compiling reports – the point of this entire project was to streamline operations for this group – and he began to explore an entirely different solution to help solve that problem. In the end, his side investigation identified a novel way to use some of the systems they were developing to deliver automation to this business unit significantly faster, also meeting the ROI target of the project before they had completed the primary deliverable.

The catalyst is comfortable with problems with no apparent solution and invites as many perspectives as possible when confronting the problem, even if they have amassed sufficient authority that a decision rests solely within their domain. They understand that a shared vision not only fosters more solutions, but increases follow through on the actual execution. The challenge facing the catalyst is that regardless of the number of feedback loops they develop and their ability to engage different stakeholders, sometimes product require breakthrough innovation and different thinking. Apple offers a striking example of apparently counter-intuitive thinking including intentionally designing products they think are cool rather than doing market research. In the end, we see the catalyst growing by building up product expertise from unique sources of feedback and eventually converging towards the path of the genius.

Two Different Paths

These archetypes present more like a series of logical stepping stones for someone to make as they progress towards being an increasingly capable product owner. Some product owners focusing on exerting authority in the decider archetype. From there they build expertise and eventually move into a genius role. Others do not have the opportunity up front to be authoritative and must opt for a diplomat archetype. The growth opportunity for them is to build authority and decision making capabilities through their growing ability to influence without explicit power, allowing them to grow into the catalyst change. When first laying this out, I didn’t mean to show a progression, nor did mean to have the two paths diverge and then converge. Is there a sixth archetype that sits between the genius and the catalyst? Is there some ideal mastery of both influence and expertise that all product owners are striving to achieve? While I certainly agree that there are always further opportunities for growth, I’m not sure I can identify an idealized end state for this journey, so for the time being I have left this spot open. It seems identifying a terminus for some perfect product owner would undermine the concept of continual learning and growth that is one of the most important things I see in successful product owners. So let me just conclude on a note that there is no one pre-ordained journey. By enumerating these archetypes I hope others in those situations can help identify their circumstances and use that awareness to their advantage.

This presentation was given at the Boston chapter of the IIBA on November 2 and covers some important common experience that good Product Owners have, 5 Archetypes of Product Owners and the growth opportunities for each of them and 4 areas to focus on to grow as a Product Owner to help not only transition to a new role, but to truly make the leap and place innovation front and center.

The slides are available here.

“Hey, man! One week down, two weeks behind…” -Kirk Lazarous, Tropic Thunder (2008)

While not a terribly serious movie, I have always enjoyed this quote from the comedy movie about the ill fated making of a war movie. The concept of being only one week into a project, and yet somehow already two weeks behind seems so absurd as to be impossible, and yet it has been the reality for a lot of projects encountered. Invariably, ambitions around projects can get so high, with so many stakeholders that our definition of success rapidly becomes so great we watch the possibility of completing our project recede into the horizon. Properly defining and then managing project success is a critical activity in any project. There are a number of excellent resources I’ve drawn upon over the years to do this. This blog post is more a compilation of my thinking on this topic threading through some of these techniques.

The Iron Triangle of Project Management

Fig 1 - The Triple Constraint

The first concept I encountered on this topic was introduced to me as a project manager. I was told to manage the triple constraint of scope, schedule, and budget. The concept is simple and quite effective. On any given project, there are fundamentally three constraints, the amount of work you’re going to deliver, the amount of money you’re willing to spend, and how long the project can run. As the name implies, these are constraints imposed upon a project, “you must finish by 12/31/12″ that will be used to determine if a project was successful or not. As the triangle image demonstrates, you can constrain on up to two dimensions, but not all three. Thus, you could engage with a project sponsor in an earnest discussion about where you would put the prioritizing focus for a given project. Let’s look at some examples to better illustrate this point.

Fig 2 - Triple Constraint Examples

The first example is a pretty straightforward single constraint where the delivery of a set number of features is most important, while schedule & budget being less important. My favorite example of this would be Blizzard entertainment, who explicitly refuses to announce release dates in advance for their games, rather insisting they build them properly and that it’s “done when it’s done“. The next example we see is a double constraint. In this case, we are prioritizing one of the two constraints over the other. A colleague working with an entertainment company offered a great example for a project like this. He had a specific team (fixed budget) and was building a new application for an upcoming sporting event. Thus, if they didn’t release by a given date, the application was worthless. In this case, the scope of what they release was actually the least important thing. Lastly we come to one of the failure models for this approach, which I would call the cop out. In this case, the sponsor basically puts the focus exactly in the middle, saying that everything is important. As you can imagine, this does not set up a project for success, as we have no flexibility along any dimensions.

From a project management point of view, I can see the value of this tool. However it leaves many questions unanswered. How important is it that we build something which is reusable and can be built upon in the future? How important is it that we invest in the team? Are there specific ROI figures that are required for this project to be viable? These are all left unanswered by such a tool. These limitations were clearly articulated when I managed to deliver a project on time, on budget, and to scope but it delivered zero business value.

Success Sliders

Rob Thomsett introduced the concept of Success Sliders as a means of exploring more than just three dimensions when considering the success of a project. This technique offer the ability to break apart that ambiguous concept called “scope”, or prioritize other dimensions like quality and team experience. Indeed, you can use this type of model to prioritize along a virtually infinite number of criteria.

Fig 3 - Success Sliders with a force ranking

Figure 4 - Score based Success Sliders

There are two common implementations I’ve seen of this technique. The first, demonstrated in Figure 3, is a force ranking. In this example, a number of stakeholders each individually had to force rank the priority of a number of different criteria such as “realize ROI”, “High Quality”, and “Deliver Project on Time”. Each individual stakeholder’s values are represented with a different color. The precise list may vary from project to project. In fact, eliciting and elaborating specific criteria can be an art in its own right. This example shows how different stakeholders show their perspective of priorities so that discrepancies can be discussed and reconciled before a project falls into duress. The ultimate goal is a single prioritization, as demonstrated in Figure 4. This is a screen shot from a tool made available by Mike Cohn on his website. This example is different in two counts. First, it represents a single set of agreed upon values. Second, it is not force ranked, but rather a budget. You can only increase the priority of one dimension by lowering the priority of another. This sort of tool, especially when used with a number of stakeholders, can be an excellent way to work through different expectations and amongst different stakeholders. However, this technique also introduces some limitations. While it does allow us to prioritize, it can’t quite distinguish what the absolute criteria is. We can focus on dimensions, but are the top 3 deal breakers? What about the top 4? There still is some ambiguity we need to cut through.

Project Constraints

Jim Highsmith proposed the use of identifying project constraints to as a way of reinventing the triple constraint. His argument is that those dimensions can be prioritized into “Fixed”, “Flexible”, and “Accept”, but those can additionally be illustrated with threshold criteria. Let’s illustrate that tool with the case of the sports application.

In this example we see that the primary constraint is schedule. The application must launch by 3/1/11. The flexible constraint is budget. In this case there is an existing team, so while there is a little bit of flexibility with the budget, this dimension shouldn’t move too much. The final constraint is that they can generally accept an application that can provide scores and game highlights. Thus we see the project team must deliver the application by the specified date while adhering to the general burn rate it currently has in order to opportunistically deliver whatever functionality it can within those constraints. As you can see with this model, whave a more precise conversation around the precise criteria.

The concept of a fixed constraint compared to a flexible constraint can be applied to the broader set of criteria identified in our earlier exercise. Let’s take a look at the situation illustrated in figure 4 using a constraint table.

What Do You Mean, “All Scope”?

Regardless of which of these more detailed techniques you may use, clearly breaking apart the “scope” dimension is critical. I have participated on too many projects that have blindly said something like, “our new system should deliver all the scope of the old system”, without really doing any analysis of exactly what that is. Indeed, I find that the question of scope can usually be broken into smaller pieces. Let’s look at some examples.

• Break by users: some applications have either a wide array of user times or clients who use an application differently. In this case, you may want to identify different dimensions to prioritize. For example, you could take an online book store and break it apart by the different roles and say something like the functionality for an online book shopper is the highest priority, as they are the ones actually buying the books. Support for wholesalers providing the books and your administrative support staff may be lower priorities for your first release.
• Break by business objectives: many projects have hard interdependencies or down stream processes that require time in order to deliver their own functionality. For example, imagine you have a web service that a customer must interface with. You may have priorities by date, based on the delivery schedule of your partner. Let’s say that you and your hypothetical partner plan to launch a new service for the holiday season, which must be live by 12/1/12 in order to take advantage of the holiday season. However, your project is not responsible for integration with the vendor. Rather you, must simply provide them with a usable web messaging interface by 11/1/12 so that they have 1 month to tie it together an test. That delivery of a usable service may be a more important criteria for the team than the actual 12/1/12 implementation date.

Let’s take one last look at a hypothetical project book store project and see how this whole thing plays out

This is More That Just Mechanics

While these exercises might seem pretty straightforward and simple, sometimes they can be quite contentious. I have encountered a number of anti-patterns when trying to use these types of techniques

• “We already know what the priority is” - frequently a project manager or sponsor may state quite simply that this level of detail is not important, because they already agree on it. In these cases, sometimes the easiest thing is to ask them simply to write it down and compare it to what some other major stakeholders think. Frequently, this will reveal enough inconsistency that they realize they need to entertain a more detailed discussion about criteria
• “But I’m the sponsor, Isn’t this simply my decision?” – sometimes project sponsors may feel offended that you are trying to democratize what they see as their decision to make. It’s very important to be clear about the point of this type of exercise and how you will use it to make decisions. Chances are, a project sponsor funding the project will ultimately make the prioritization decisions and define the final criteria. The point of an exercise like success sliders or constraints isn’t to do that for them, but rather to surface the expectations and perspectives of other stakeholders so that the sponsor can make the best informed decision. If the dev manager says that reusability is very important for the success of this project, that very well may at least be worth entertaining the discussion to understand why. There’s a chance they may know something the sponsor doesn’t and the discussion will help lead the group to a better decision.
• “This is ALL important” – some groups of stakeholders may be unable to parse more detailed criteria like in some of our later examples in this article. Rather, they may have a very hard time saying precisely what is or isn’t critical for success. This is usually where a force ranking, like in FIgure 3, can be most valuable. Before even engaging in a discussion about which dimensions are truly must have, or even the precise dimensions, stakeholders must simply prioritize in a force ranked order.
• “We don’t want to preemptively surrender scope” – when engaging in this exercise with groups that have little trust, its quite possible the business may reject this type of exercise feeling like the development team is simply trying to get them to give up scope or lower expectations. The point of this exercise isn’t so much to surround things; ideally, we’d like all dimensions. The point of the prioritization is to do responsible project management. We want to know what’s truly important before we start the project so that people can make informed decisions along the way ad new can manage risk so that if things do go wrong, we are able to maximize the chance it won’t impact our primary success criteria.

There is a classic business problem regaled in business school where the management company of a large tower is struggling to deal with complaints about wait times for their elevators. The company brought in consultants to investigate adding additional elevators and other expensive solutions. Then, brilliantly, someone re-frames the problem from an engineering one to a psychological one. It’s not that the elevators were slow, but rather people perceived they were waiting too long. If you solve the perception problem, then people won’t complain. What did they do? They put up mirrors in the elevator lobbies so that people would be distracted when waiting for the elevator. This novel approach effectively resolved the problem for a fraction of the cost that would have been required had this been viewed purely through the lens of an engineering problem. This is a perfect example of an Agile project leveraging learning in the middle of the project to fundamentally change directions to deliver a better result.

Within software development, we see the same challenge: depending on how we frame a requirement or user need, we may end up with a very different project. Herein lies a key challenge when running an Agile project, if your requirements are not captured in a flexible manner, you will sacrifice a significant amount of value. Had a project been chartered to increase the elevator capacity, as opposed to fix the complaints about wait time, the team would have been unable solve the problem simply by putting in mirrors. Indeed, they may have become contractually obligated to deliver more elevator capacity, locking the customer into a costly solution they did not need.

It’s not that the traditional Work Breakdown Structure is incompatible with Agile projects, but rather that the way we organize our work and use such a structure is crucial to how successful our project is. Let’s take a closer look at how we manage requirements within an Agile project. There are three key concepts we will walk through in detail: requirements should be small enough to fit into a short period of time, they should be prioritized, and they should be based on the user’s perspective.

Align your Work with the Business

By definition, an Agile project is meant to be executed in close alignment with the customer’s needs. Not only should it begin with this alignment, but it should be managed in an adaptive manner such that the project can adjust based on insights gained and changing circumstances. How exactly do we communicate the scope of work and requirements in a fashion complimentary to our goal of being adaptive? Enter the product backlog!

Originating from Scrum, the product backlog is a prioritized list of things needed for the product being built. In its simplest form, this could simply be a force ranked list of features or capabilities. Here is my simple list of features needed for my online book store. You will see that I have identified 6 features and placed them in a forced rank order.

A sample prioritized backlog for an online book store

I need to reiterate that when we say prioritized, we don’t mean “high”, “medium”, and “low”. Rather, our goal is an actual ranking, where item one is the single most important thing, item two is the next most important thing, and so on. Prioritizing into buckets leads to games. Have you ever had a project where 90% of the features were “must have”? Me too. If everything is a high priority, then nothing truly is. In fact, it is irresponsible project management. If I can’t tell what features are truly mission critical, I can’t make sure they are properly completed above other features that may not be as important. If you take nothing else from this article, get your customer to force rank their requirements, the clarity you get from this exercise will be invaluable.

The priority of the backlog is critical, as it drives the order in which features are delivered. Those on the top will be undertaken first, and those on the bottom will be deferred until all higher priority items are completed. Some people will refer to these items as “sushi slices” of functionality, where we define a piece of work that consists of a little bite of each type of task, as opposed to defining work items that are large and homogeneous. The image below should help you see what we mean when we say a feature that requires a little bit of each type of task.

Cycles of work in a waterfall and Agile project

This backlog is generally managed by the product owner or whoever is representing the business, and may be changed at any time. The general rule is that the business may change the priority of any piece of work that has not yet been started. Agile projects can benefit immediately from this type of prioritized list by using the progressive elaboration. This allows the team to focus energy on those items that are highest priority and will be done next. Those items that are of much lower priority can remain ambiguous. Simply stated, we don’t need to come up with the requirements or design for the entire system before we can proceed. This allows us to begin building and validating small pieces of functionality rapidly.

Define Work Items Small Enough to Complete in Under Two Weeks

Most Agile projects use some sort of time boxed iteration, where a team will take on work that it can to complete within a time horizon of about two weeks. Teams need to be proficient breaking up large requirements into progressively smaller capabilities to deliver. Smaller pieces of work improve testability and flexibility when prioritizing features. The Standish Group’s 2002 CHAOS report observed that users reported rarely or never using nearly 2/3rd of all features delivered in their systems. Breaking apart requirements into smaller pieces allows us to deliver the 1/3rd that are truly important without being tied to those that are less important.

Building on my earlier example, we may see that something like searching for a book is quite large once you begin to think about all the different dimensions upon which you may run a search. Here I have shown it being broken into four distinct features. What you will also notice is that they are not all of the same priority. Searching by ISBN was actually put on the very bottom of the list and represents one of those features that probably shouldn’t be built, but if we took on all of search as a single work item, very well may have bloated our product.

Sample book store backlog with search capabilities split into smaller features

Frame Work Items on Value for the Business

So now we have an ordered list of things to build for our product that are small enough we could fit them within an iteration of two weeks. That’s great, but it’s not enough. Simply having a list isn’t enough, think back to our elevator problem. They could have easily put together a list that included things like, “run elevators more efficiently” and “add more elevators”. While they could work through those items individually, we’re still focused on a single perspective. Teams frequently fall into this trap, they focus their backlog either on tasks to do or on technical components. Considering that many IT teams are led by project managers or tech leads, its natural they would bring their own bias towards how they look at a project. The challenge being, this is not how the customer thinks about the problem. Rather, we want our work to be focused on the experience of a user, because those are the outcomes we’re trying to influence. Focus on the user also gives us maximum flexibility to plan our work around what the customer really cares about, the experience and outcomes.

There are several different techniques you could use to capture requirements from a user perspective. For purposes of this discussion, let’s talk about User Stories, a practice from eXtreme Programming which is probably also the most commonly used within Agile software development. User Stories are a mechanism for framing requirements from the point of view of a specific user. They contain two parts, a narrative and acceptance criteria. The narrative contains an actor, an action and a justification and the acceptance criteria contains the criteria the business will test to approve a given story. While this structure is pretty useful, the important concept is that we’ve framed each feature as an experience, they are technically agnostic and they each have a business justification, what I would call the “so that” of the story.

For co-located teams, User Stories are designed to be lightweight enough that each one is put on a note card and stuck to a wall. In fact, User Stories are meant to be a placeholder for a conversation,  thus they give enough information for us to get the general idea, but encourage us to go speak to the business to make sure we completely understand. The acceptance criteria helps us nail down very specific criteria we will use for saying a piece of work is complete. Building upon our earlier example, here are the first two features written as user stories

Sample User Stories from book store backlog

Reconciling the Product Backlog with the Work Breakdown Structure

None of what we have walked through so far is in conflict with what the PMBOK® guide says about work breakdown structures. In fact, a review of the guide will show the following definition:

“The work breakdown structure (WBS) is a deliverable-oriented hierarchical decomposition of the work to be executed by the project team to accomplish the project objectives and create the required deliverables.” (PMI Project Body of Knowledge® 4th Edition, . 146)

The only difference is that with an Agile project, we expand our horizon of project goals and try to align them directly with the outcomes of the business. Thus, the goal of an Agile project won’t be to implement the XYZ system, but should rather be based on a specific outcome or experience. Beyond that, the same concepts apply. WBS can be progressively decomposed to smaller and smaller deliverables, much like we broke apart the book search feature. The PMBOK® supports the idea of progressive elaboration, so that we have invested a lot of time and detail into those items coming up next, and not a lot on those that won’t be addressed for a long time.

Empowering the Business

So those are the mechanics we use to manage scope and requirements. The real power comes in the way that a business partner can use them to maximize their own success. This article doesn’t have sufficient space to fully explore this topic, but let me offer you a few strategies used by effective product owners and project managers to get the most value out of their agile teams.

• Prioritize on business value – the team can simply start on the most valuable item and keep working until the law of diminishing returns kicks in and it is not worth working on anything else. This strategy will help a business build a lean product without extraneous features.
• Prioritize to maximize learning – in some situations where the solution is not fully understood, teams will pursue a strategy of prioritizing work in order to maximize their learning within the domain. You could almost think of this as the scientific method, where a product owner may establish a hypothesis about how to solve a problem, test it and then adjust the theory.
• Prioritize to minimize risk – good project managers know that if you’re going to fail, fail early. Working with the business, some teams will adjust the order of work in order to deliver those features that are highest risk first. The strategy being that if the project is to fail or encounter severe problems, these are the things that will cause problems. By doing them first, the team has the most amount of time to respond and risks can be confronted and removed earlier from the project.

These strategies aren’t mutually exclusive; teams frequently start with one and transition to another. Many teams will begin by prioritizing to reduce risk and maximize learning, and then as the domain is better known and less uncertain, adjust to a priority based on business value.

Conclusion

A product backlog doesn’t make you Agile, but a poorly designed and managed one will sacrifice a significant amount of potential value from an Agile team. Thus we see that a well used product backlog allows the team to properly align their work around the goals and objectives of the business. It brings that business into a closer partnership by giving them unprecedented transparency into what is being done and in what order. Lastly, it encourages collaboration. Each piece of work the team undertakes is tied to business outcomes and justification, inviting the business to partner much closer with the team working to deliver the project. Ultimately our goal is to deliver value for our customer, the more we can empower them to make decisions best representing their interests, the more value we can deliver.

Old Models Don’t Hold Up

When looking at organizations trying to manage a portfolio of work with Agile teams, we frequently find one of two common patterns that no longer supports the organization

• Each item run as a project: I’ve written before about the pull some organizations feel to solve all their projects with individual projects. Indeed the very success of prior projects can become their undoing as everything becomes a project, and stakeholders then expect to see all projects start so they can see progress towards their goals. This ends up leading to a mental model where executives may love Agile teams, but they feel that they couldn’t run their entire portfolio using it, because they don’t have 20-30 teams to manage the 20-30 projects they are running at any given time
• Large Project Broken into Silos: some organization undertake the exact opposite approach. Rather than having numerous tiny projects, they run large, monolithic ones that encompass more work than a small group of individuals could ever achieve. In these cases, we frequently see smaller build teams within the overall program, and they are frequently crudely cleaved around a functionality, application, or technology stack. This can introduce many problems, as teams may not be able to individually deliver much work, small sizes of each can introduce lots of variability and the cleaving of work across teams may also mean that the most valuable stuff is not confronted first (sure, we don’t need to build the more for the XYZ component right now, but that’s what the XYZ team is tasked with doing!)

Building Blocks within a Pipeline

Thus the challenge becomes managing a pipeline whereby work can be fed to teams in digestible sizes that are properly prioritized and elaborated. There are a number of different approaches and workflows that follow an idea from conception to prioritization and delivery by a team. Looking at these different flows it occurred to me that this is really just workflow encompassing some combination of a series of transformation steps. At the risk of being accused by my coworkers of “going meta” too much, I figured it would be worth looking each of these steps individually.

A good program manager can use these steps like building blocks to assemble a pipeline process to offer the necessary transformations to keep their project teams happily working on the most important work at a sustainable pace. Indeed my experience has shown that many of the lean concepts like managing work in process, cycle time and value stream apply quite nicely.

Prioritized List

Prioritization Step

Purpose: The prioritized list ranks work according to some measure such as value, time needed, or even a weighted importance across several dimensions.

Where is it used: within Agile teams, prioritized lists are probably one of the most commonly used portfolio management tools. Teams have a backlog of prioritized work to be taken on over time. When looking at a large portfolio, the same concept applies, by prioritizing work, we can figure out what should be sent through the pipe first. This tool should be at the very beginning of a product pipeline.

Common Applications: business value or expected order of delivery are probably the two most common lenses used for prioritizing work, but there are a number of other techniques ranging from the wisdom to of a product owner to dynamic group decision models such as multi-voting or even Innovation Games® such as Buy a Feature.

Filter

Filter Step

Purpose: Removes items from the backlog that do not meet a certain criteria (strategic alignment, value / cost, etc.)

Where is it used: frequently used at the beginning of a pipeline to prevent clearly extraneous things from entering. Also used after work is further elaborated or decomposed to remove things that, upon further elaboration, don’t meet a certain criteria or to eliminate parts of a larger item that was just decomposed

Common Applications: The most common applications of a filter would be some sort of ROI or strategic alignment tool where projects or their components must be justified by in some way aligning with goals or meeting an expected ROI threshold. As such, we frequently would see a filter after some sort of elaboration step.

Decomposition

Decomposition Step

Purpose: Breaks apart large deliverables into smaller ones that are more likely to fit within a team’s backlog or iteration

Where is it used: decompositions are generally used once a piece of work has made it through some level of due diligence and prioritization. The point is that this work is most likely to be undertaken, at least in part. While breaking deliverables into smaller pieces requires analysis work and carrying costs, it helps make estimating more accurate, provides smaller pieces of work that can be fed into teams and allows a better prioritization of features so that some sub-components can be delivered faster and wasteful gold plating can be removed.

Common Applications: Story slicing or the functional decomposition of large epics into user stories are common approaches for breaking apart stories. Story mapping is another incredibly powerful technique here for breaking apart what, at first blush, appeared to be an atomic component.

Elaboration & Sizing

Elaboration Step

Purpose: I am including these two steps together because my experience has been that they are invariably tied together. Elaborating deliverables is required to estimate how big they are, and the act of estimating invariably ferrets out numerous details of what would need to be delivered.

Where is it used: this is a fundamental step of any pipeline taking advantage of progressive elaboration, where we add more detail as time goes on. Indeed, this concept is even used in most traditional projects where we move from high level business requirements, to functional requirements, to system requirements and finally a design. Likewise, elaboration and sizing may happen several times within a portfolio process where we add “just enough” detail to determine a high level size in order to know if a feature could fit in a 3 month release or not. As it gets closer to delivery within a release, it may need to be further elaborated so that a team can take it or some piece of it into a specific sprint.

Common Applications: There are numerous forms of parametric sizing ranging from planning poker to more abstract ones like t-shirt sizes or, one of my personal favorites, dog sizes – really, you haven’t led a planning session until you hear people argue about whether or not a piece of work is a “standard poodle” or a “great dane”. Regardless, most techniques revolve around a similar concept, discuss the details of a piece of work enough compare it to other similarly sized pieces of work.

Allocation Grid

Allocation Step

Purpose: Many pipelines need a level of evenness of flow. The idea of a grid is to allocate a certain percentage of the pipe to different types of work in order to ensure that one things doesn’t overwhelm the others. This could be allocations of the pipe to different initiatives based on funding, thereby ensuring that each funding sponsor gets “their share” of the development output. It could also be used to implement the lean concept of leveling the work in order to ensure that work flows consistently. Lastly, it may be used to ensure we don’t accumulate technical debt or neglect important parts of our portfolio such as maintaining current systems or technical infrastructure.

Where is it used: Grids are frequently used once work has been broken into smaller pieces and has been prioritized.

Common Applications: It effectively can split one backlog into multiple. For example, if a program allocates 20% of its through put to maintenance work, then the top maintenance items will be pulled from the backlog to fill the 20% allocation, even if they are not top on the list. It can also be used to level work in the case where a team or teams must manage multiple types of work such as stories that involve only UI or stories entirely focused on business logic. Teams may even have specific allocations for technical improvements or refactoring in order to ensure they have some protected allocation of time to eliminate technical debt and run at a sustainable pace.

Putting it Together

I’m finding that most portfolio management processes can be found using combinations of these different building blocks, with each step a transformation along a value stream. Just like when using a task board, I would expect to see the precise flow over time as people mature, but let’s take a look at a couple example to demonstrate the point.

Prototypical Path

Let’s start with our challenge of too many requests. Traditionally, each request may very well become it’s own project requiring overhead and all the accompanying costs. This model below demonstrates how we would put request through a methodical process to elaborate, prioritize and limit the work that gets to the team. At the end of the process, we have a single consolidated backlog from which an individual team could pull work. The net effect is that the team would have the highest importance work across the entire portfolio, allowing the organization to reconcile the work of a program with a single team.

So here we see a development pipe walking through the following steps

1. Requests come in through multiple sources
2. Immediately apply a strategic filter and try to remove any requests that are not important
3. For those requests with merit, break them down into smaller stories
4. Prioritize all the decomposed stories, so that you have an individualized list of priority stories / features across all requests
5. Filter out those low value features or stories
6. Add detail and estimate the size of the higher priority decomposed stories
7. Prioritize those stories based on their net value (benefit – cost)
8. Allocate 25% of the pipe’s through put to each of the four initiatives

The key step I’m missing here is feedback, and as my coworker Mike would remind me, we should have feedback along every single step of the way to ensure that what we’re doing is indeed valuable and moving us towards our strategic goals. I’m not sure I’ve figure out how to fit that into a diagram like this, but I’m still working on it. At this point, I’m just trying to articulate a common set of components I may use when helping a client define and manage their program process for managing multiple requests. Like a Kanban system, right now I’m just trying to offer a tool for visualizing the flow, if we can’t see it, we can’t do anything.

What’s Next?

I would still consider this a work in process, but I would welcome feedback from others. Are there other critical transformation steps you see missing? Does a diagram like the one I laid out at clarity or make such a process more confusing? I suspect I’ll be writing a bit more about this soon, but I hope you enjoy the idea as I mature it further and thanks again to my colleagues for helping me to develop some of these ideas.

Why Do We Run Too Many Projects?

My colleague Mike Dwyer refers to Scrum as a Silver Mirror. By this he means that, while it won’t solve all your problems like a mythical silver bullet, it will present you with a level of reflection such that you can identify and confront your problems. To that point, my perspective has been that organizations struggling to launch Agile projects because they keep trying to run too many projects are suffering from a larger issue.

• Lack of Agility
  You may find this counter-intuitive, but I would posit that many traditional organizations use broad project portfolios as a means of achieving agility. With the heavy ramp up costs of any given project, having multiple options to respond to change. We can ramp up or down our energy on any given project based on changing circumstances or priority. While it’s not terribly flexible, it does offer some level of adaptability. However, it comes at the price of running all these projects at the same time, which is a very steep tax when you start adding up the cost of multiple status reports, daily or weekly meetings, code branches, design documents and any other overhead required for running your projects.
• Lack of Impediment Removal
  I remember before I formally began my foray into Agile project management being frustrated with a manager about a policy of putting everyone on 2+ projects at any given point in time.  After some lengthy discussion, he finally explained to me that by putting my on multiple projects, even though it would extend the delivery date of both, it would ensure I was productive. For if one project became impeded, I could simply focus my energy on the other. This may be true, but it completely hides the fact that we have issues with a project. It also defer risk by redirecting effort to the less impeded area, creating a false sense of a rate of progress.
• Lack of Priority
  In some organizations, especially with many influential stakeholders, prioritization can be difficult. I have seen organizations where prioritization is implicitly thrust upon the development teams. The business basically asks for everything at the beginning of the year and then the technology partners respond when each item will deliver. Of course in these cases, the vice of no prioritization can be mixed with a desire to “show progress” on all projects with awful results. This is where we see more projects run than people so that some portfolio dashboard can show progress on all these different capabilities, but unfortunately it is really just an illusion.

I enumerate this list not to justify behavior, but to empathize with the conundrums facing many project managers. In fact, I believe that – in a strange way – the propensity to solve every problem with a project is a testament to the success of projects within large organizations. A generation of business people have seen careful project management get things done in chaotic and uncontrolled environments. Good project manager have leaped into the void of poor organizational impediment removal and prioritization and responded to these challenges by starting more projects. The only problem is that slowly the added cost of running these projects undermines our ability to do anything. This is a problem more projects can’t solve. As Albert Einstein famously observed, “Problems cannot be solved by the same level of thinking that created them.”

This level of thinking has reached it’s limit. We’ve gotten to the point where organizations deify projects to the detriment of teams and people. How many of you have seen organizations that will never stop a project, but they have not problem asking someone to work 25% or less on multiple projects across a portfolio? Or companies that talk about the value of teams, but keep sloshing people – probably referred to as “resources” – between projects on a regular basis? Indeed when we see this situations, where people feel totally at the mercy of the projects to which they are assigned, is it any doubt why people don’t self organize into hyper-productive teams?

Kill Your Projects, Get More Work Done

Several of my coworkers have toyed with the concept of “killing projects”. This would be the natural progression of Derek’s earlier point about aligning projects to teams. Its a great idea, but in many organizations, we may never be able to realistically say no to several projects in order to focus on a few. There may be numerous stakeholders with very real concerns that demand action immediately. Or perhaps we are simply within the bowels of a large organization and we are presently unable to influence our stakeholders to the point where they would pick one project over another. In the old days, we would slice apart people to make whole projects. Today, I’m proposing that we take the knife and apply it to projects instead. Rather than try to run multiple projects.

By leveraging a single product owner or product team to triage and break apart the work, we can feed what would have been multiple projects to a single team. This frees the team to figure out the best way to balance the work. If they are operating according the practices of producing production ready code every iteration, then they will also be able to deliver value to all stakeholders, something that would go a long way in those low trust environments where teams needs to show progress across all fronts. The other benefit is that by putting all requirements through a single funnel, they bring the different stakeholders together to discuss what’s ultimately most important.

They don’t need to work together. I’ve seen teams allocate the capacity of there throughput based on funding levels. For example, imagine that three projects were approved: Project A funded for $250,000, Project B for $350,000, and Project C for $400,000. This would lead to a single team funded at $1,000,000. We could easily allocate the team’s velocity by funding levels, thus each team would get 25%, 35% and 45% of the points delivered for each sprint or throughput on a kanban system. Either way, stakeholders would be able to get what they want based on their funding, but hopefully putting them together would lead to the model described by David Anderson (2010) where people would move through phases of first selecting work based on their allocation, then moving to some sort of democratic model and eventually moving to a model where they dialog about what’s most important for the organization next. This model could even be scaled to multiple teams, where they would pool requirements across different projects and figure out what they can build for the next release. These joint planning sessions are usually quite energetic with multiple product owners, teams, and stakeholders operating in breakouts within a large room to trade stories, make prioritization decisions and plan the work.

Ultimately, we are proposing a model to help you confront the issues of your organization. In order to be successful with any type of project portfolio strategy, we will need to be couragous enough to look into that silver mirror and address our shortcomings. It is the fact that the multiple project strategy hides these issues that is most dangerous. Not only does it offer little in the means of helping us address impediments or answer prioritization questions, it conspires with us to brush all of those things under the carpet and simply look at the problem as one of planning and organization. Those are safer topics to be sure, but we do ourselves and our organization a disservice by not confronting the real challenges.

The Challenge

So we have problems to solve, systems to build, and domains to explore. These are quite important goals, and one where Scrum is largely silent. I’m not saying that Scrum doesn’t work, so much as Scrum assumes we have this benevolent and omniscient Product Owner. The Product Owner decided what’s important, they maintain the vision of the product and they are the “Single wring-able neck”. Simultaneously a business expert, visionary, and pragmatic manager, the Product Owner is able to articulate a perfect vision of a product which is now incumbent upon our Agile team to deliver. Sure we may adjust as we start to build it out, but that will be based on the Product Owner’s wisdom and foresight. I have met only a handful of people who fit this bill. Frequently I find that the person tasked with managing the backlog for a team is a bright, capable, but very mortal person.

• There is no definitive expert
  In many modern, complex business domains, no one person could hold a view of the entire problem. Chances are that if they see the whole thing, they are not in the trenches and only have an abstract view of what’s going on. Conversely, if they are on the front lines, they are so consumed by details they may not fully see larger patterns playing out over longer time horizons. Such is one of our key challenges: the act of determining “what” to build is more complex than cracking open some really smart person’s head and removing the collected wisdom within it. A further challenge is that even if you have an expert who perfectly grasps the domain, a growing body of research shows they will not be able to easily articulate that understanding to others. The very knowledge that helps us provides the expert with a context unavailable to others; something that appears patently obvious to that person may sound like Greek to others. The Heath brother’s dubbed this the “curse of knowledge“. For a good example, imagine a consummate fan of Star Wars, now put that person in a room with someone who has never seen the movies. Now imagine the awkward conversation that would ensure as the Star Wars “expert” tried to convey the key points of the movies.  A dependence on an enlightened expert may simply degrade into the team being told to “do this” without really understanding what’s going on, preventing them from offering valuable feedback and being able validating what they build.
• Real Value is Derived from Interaction Between People
  So if our systems are too complex for any one person to fully understand, we also don’t necessarily generate the value by simply stringing together a lot of narratives. Rather, fundamental value is derived from the interplay and increased perspective we get by having multiple people interact. One of the most powerful demonstrations of this is the “wilderness survival” exercises. The game comes in different varieties, but the general structure is this: you are stranded remotely in a forest / arctic tundra / tropical island (my favorite) and you must prioritize from a list of possible things to do which ones you would do and in what order. First set your priority as an individual, then do it as a group. After doing this, each group gets the correct answers based on wildlife survival experts. What’s most interesting is that in almost every version of this game, the group rankings are better than every single individuals decisions. The act of working together produced substantively better ideas than simply leaning on one expert or amalgamating everyone’s response. So we see that a huge part of the value discovery within analysis from a project emerges from the interplay of multiple people simultaneously playing with the same concept.
• Seeing the anatomy of something isn’t sufficient to understand it
  A dinosaur fan growing up, I was recently heart-broken to find out that the triceratops never existed. Apparently, archeologists looking at the fossil record mistook what we now believe to be juvenile torosauruses to be an entirely different species. Think about that; for over 120 years – the triceratops was first documented in 1889 -  we mistook a juvenile of one species for an entirely different animal because we couldn’t understand it within the context of the bigger picture. Of course, we can forgive our archeologist friends. They can only make observations of these creatures by examining their remains in a sterile environment completely separated from where they actually lived and interacted. But the question is, why would we ever want to do this with modern systems where we can touch, feel and play with them. My favorite, more business related example, is New Coke.
  Inspired by internal taste tests showing people preferred the taste of the sweeter Pepsi, Coke sought to change it’s flavor in order to win blind taste tests. Of course, people don’t blindly sip soda, rather they drink a whole glass. While a majority of people preferred Pepsi’s taste for the first sip, once you drank a whole glass, more people preferred Coke (Gladwell, 2007). The way you ask your customers what they want is critically important.

Why Games?

Nearly 25 years later, we find ourselves continuing to face the challenge articulated by Fred Brooks, how do we know WHAT to build. Agile software development has added huge value in this domain by extolling the values of progressive elaboration, emergent design and iterative development. Still, we need somewhere to start and we need ideas to build upon. How do we maximize the value of the ideas we feed into our development teams? If our goal is to maximize ideas and generate innovations, we want an environment conducive to those points. At the risk of playing pop psychologist, let me walk you into the field of positive psychology. Barbara Fredrickson, in her pivotal paper, “What Good are Positive Emotions” observed that the bulk of research into human psychology focused on ailments and maladies. For the majority of its existence, the profession has been fixated on understanding the ways in which the human mind breaks down while neglecting what happens when we’re happy. Specifically, she identified several key benefits to people when they are experiencing positive emotions. We are all familiar with the idea of “fight or flight”, the moment of stress or fear when the human mind narrows all options down to those two and makes a split decision. We’re probably also familiar with the cliched comment that this probably was very valuable when man had to deal with the likes of saber toothed tigers.

What people frequently overlook is what happens to people when their emotions move to the other side of the spectrum. While anger and fear narrow our options, positive emotions broaden them. People in that state are more likely to experiment with new things, make more associations, and even solve problems faster! This is not to mention the massive amounts of social capital people build when playing with others, a major factor for personal and project success. It is too bad that many people in the business world continue to suffer with the bias that if people are having fun, they aren’t working hard or fast enough. Indeed, if the growing body of knowledge on human emotional states is any indicator, a high stress environment will indeed cause people to make poorly informed decisions as a rapid rate, thereby getting us bad software faster. I don’t think that is really our goal.

Innovation Games® to Answer Brooks

Earlier this month, I had the opportunity to attend Luke Hohmann’s Innovation Game® training class, and I must confess it was a bit of an epiphany for me. For those of you following this blog, you’ve probably been noticing a “serious play” theme emerging. If that didn’t do it, the fact that I’ve helped organize two Agile Games conferences may have been the tell. In spite of this personal journey, getting to use these techniques with a group of other professionals was a truly trans-formative experience for me.

One of the key take-aways for me was that serious play, like any form of analysis requires a significant amount of forethought in order to be successful. I have included a very simplified and roughly linear flow showing the progression we used as we dealt with different case studies.

The entire class was based on case studies where we would discuss an abstract problem and then figure out how we would go about solving it. I found it particularly interesting how we began the class by reading a case and jumping to the orange steps where we were discussing what type of game we would like without methodically working through understanding our problem and the question we were trying to get answered. Much like “The Hitchhiker’s Guide to the Galaxy“, we came to appreciate that the answers you get are only valuable  if you know the question. (For those of you without a refined appreciation for classics such as “The Hitchhiker’s Guide to the Galaxy”, the entire first book revolves around a super intelligent race that has determined the answer to the ultimate question of life is “42″, but then they realize they don’t know what the actual question is.)

This is probably one of the big challenges in the Agile Software development field right now. With more people coming to appreciate the value of serious, collaborative play, the answer to more problems is “play a game”. However, if we haven’t planned the right game, and we don’t understand what we’re going to do with the information, that we may be wasting everyone’s time. So what type of questions could we get answered by a game? Well, let’s stick with Mr. Brook’s problem, what is the correct thing to build?

Let me offer a real example I’m going through right now, the PMI Agile Community of Practice. Within the Project Management Institute (PMI), I have worked with a number of committed volunteers to launch a virtual community – now 8,000 strong – of project managers looking to better align their craft with the values and practices espoused in the Agile Manifesto. The PMI has traditionally leveraged geographic groups, like local chapters, to interact with their members, and an entirely “virtual” community is something new for them. So what do our members want? What is value to them? How do we make sure we build out something of lasting value and don’t just replicated the various other online “communities” in this space?

In this case, the key question for me was “what do we want to do next?”. Actually, that didn’t quite cut it for me, it wasn’t until we moved to the next question that I got clarity: “what will you do with the information”. Well, we’re planning for the 2011 year, we are looking to figure out what initiatives and programs we should launch for the membership this coming year. That means, the key question we need answered is “what PROJECTS should we run within the community”, thus our rounds of play need to move us to concrete projects we can decide on.

Imagine our creative process being two funnels. First, we want to generate a lot of ideas. Part of picking the right thing, is making sure that we’re considering it in the first place. Once we have the ideas, we can craft them into concrete proposals and finally we can make a prioritization decision. At the end of this process, we should have a series of concrete proposals that we can prioritize for delivery. In order to reach this outcome, we selected 2 rounds of play. For the first round, we selected the game “Prune the Product Tree“, where people would use our proposed image of two trees growing together, one for the PMI community and one for Agile, and their job was to decorate it with apples representing the fruits they would hope to get from these trees going together. Armed with a series of ideas, we would then draft and estimate the relative size of a series of proposed initiatives the community could undertake.

In order to prioritize those decisions, we wanted to hear from as many people as possible, and we really wanted a decision that would reflect a whole. Thus, something like simple voting or a survey would not suffice. We selected the game “Buy a Feature“, where people jointly bid on the items they want to collaboratively purchase a single set of features. So there you have it, a 2-step collaborative process whereby we let our members propose ideas, organize & estimate them, and then have the membership jointly vote on complete packages to inform our priority. Now, there is one missing step that we’re not using a game to resolve. Namely the synthesis of those ideas on people’s product trees into concrete projects they can “buy”. Perhaps this will be something to explore for the next time we undertake an initiative like this. This has still been a relatively new process for me, and as we conduct the PMI Agile Member Engagement Initiative, I’m sure we’re going to have a number of lessons learned I will be happy to share at that point in time.

This calls to mind Stephen Colbert’s infamous criticism of modern media

The president makes decisions; he’s the decider. The press secretary announces those decisions, and you people of the press type those decisions down.

Make, announce, type. Put them through a spell check and go home. Get to know your family again. Make love to your wife. Write that novel you got kicking around in your head. You know, the one about the intrepid Washington reporter with the courage to stand up to the administration. You know–fiction.

–Stephen Colbert, 2004 White House Correspondents Dinner

Business analysts aren’t reporters, but I see a similar dynamic. We see and touch the final document – or article in the case of a reporter – but that isn’t the real value that was created. The real value was the uncovering of details, careful synthesis of multiple points of view and creation of new perspectives and evaluations of the environment. Indeed, we don’t want analysts to be stenographers, but rather to push back and challenge those people saying what they want in order to help them better articulate and understand it themselves.

The Stenographer Analyst

In the world of the “Stenographer Analyst”, the worth of an analyst is directly related to the size, complexity and general impressiveness of their requirements documents. Sadly, experience tells us that the more “impressive” the requirements look, the less people will actually read them. Thus, we create a system of local optimization where an analyst is motivated to create a big document that may actually undermine project success. However, it may protect them or the team later if the project gets into a fight about what’s “in scope” for the project. I think this is the essence of what the writers of the Agile Manifesto meant when they said “collaboration over contract negotiation”. Intuitively, we can sense this is wrong. If we could align our objectives, we could spend time and energy building out the best idea, rather than fighting with each other.

However, many of us do not yet live in this world. We are surrounded by managers who want to validate that everyone’s doing a good job, and they want to see proof that people are working hard. This is where the idea of the big document is so seductive to an analyst. It is (relatively) easy to write a document detailing a process or a system. It does require work, but producing a thorough analysis and putting it into a document is a very achievable thing that can be done and is within the sphere of control of an individual business analyst. What we’re doing when we talk about Agile Software Development is asking that analyst to let go of their document and cast their lot with the entire team and the business; to agree that we will all measure our success based on business outcomes. No more can we say, “it wasn’t my fault, I clearly documented the requirements”. Indeed, we are now responsible not only for capturing that information, but for distilling it, and communicating it. This is incredibly powerful, but also scary. We can be part of something much grander, we are working towards achieving real meaningful outcomes, but on the other hand it also means we will be held accountable for achieving real outcomes. We will only succeed if those around us do as well, we aren’t individual masters of our own destiny. It’s easy to talk about collaboration and team work, but this is the real thing: we can’t do it on our own and if things don’t work out, there is no discrete “deliverable” to fall back upon.

The Value of an Analyst

So in this environment, how does an analyst help out? Presumably the business analyst is not the actual customer, and now we are inviting those end users to work with us directly. Did we just make the analyst role obsolete? Not at all. As most analysts know, good business or systems analysis is much more than simply asking, “what do you want the system to do”. Inviting end users to come in and share their perspective doesn’t immediately solve this problem. I recall working with one group of call center operators. They were happy to be invited to our Agile project and keen to give advice. They were so excited they began rattling off requirements, “would like the system to have a button that I can click which will add up these three columns and apply our configurable business logic”, one person said.

I slowed them down, and walked them through user stories and that we wanted to capture our requirements based on a user experience. I put up the format “as a…, I want to… so that…” to reinforce the concept. The woman took it all in, thought for a moment and then ventured her first user story, “as a user of the system, I want the system to have a button that I can click which will add up these three columns and apply our configurable business logic, so that we can process transactions according to our procedures.”

This game played out for another hour as we tried to crack exactly what the summing of these attributes would do, what made up the company’s “configurable business logic” and why any of this was valuable. As the conversation matured, the manager for the department came to realize a lot of what her people were asking for was not really consistent with the stated goals of that call center. The conversation eventually moved into a discussion of business processes, which were important and how we could potentially improve them with a new system. Had we simply viewed the role of the analyst as a stenographer, we would have ended up with some nicely typed requirements that would have reinforced the organizations currently flawed system and failed to help the business understand how their operation was working.

But that is only half of the value we can get from facilitated and interactive sessions like these. Let me offer you another example, I was working on a different project at a different company where we were building a rules engine that would measure and report on a fund’s adherence to different regulatory rules. Periodically the business would come and ask for new attributes or functions with which they could build new rules. I was still fairly new at the company, and when dealing with one request that we got for a new function and supporting fields, I naively asked, “what do you want to do with this new functionality”. The business partner was taken back by the question, apparently they weren’t used to having to explain things on that level, but it came out there was a specific test they wanted to run. As we talked through what this test was, we realized they could replicate that test using some attributes and functions already existing within the system. We were able to end a 3 month project in a one hour meeting and implement the change that day for them, because we were able to re-frame the conversation.

I’m not arguing that these are activities which other analysts don’t do today, indeed this is the bread and butter of good business & systems analysis. It’s not about writing down what people say, it’s about helping them understand what they really want and broadening the perspective by bringing in more stakeholders. That is a very real activity, and getting rid of a big document doesn’t make that activity go away. In fact, it should help make it easier by changing the system into one that is more conducive for us to interact and explore an uncertain domain.

What Prevents Us From Getting to Good Solutions

Hand Offs

As children, most people get an opportunity to play the game “telephone”, whereby one person starts a message by whispering it into the ear of the person next to them. The receiver can not question what they heard, they must simply relay it on to the next person, until the message has traveled through all of the participants and the last person announces what they heard, which invariably is profoundly different from what the original person said. In spite of this well known dynamic, many projects operating in technical silos insist that we play this game with our projects.

Certainly Agile practices help here, as they tend to break down the large silos and force people to work closer together with smaller pieces of functionality. Still, the idea of walking through a decomposition of business requirements to detailed requirements, to designs, to tests introduces massive risk that something can be lost in the numerous translation. Even in small chunks, we are basically giving each audience the chance to frame the problem according to their perspective. The business will frame the problem through their eyes, the analyst will craft use cases or flow charts, the developers will build detailed designs, and the testers will identify concrete scenarios to validate the functionality. Of course we invite others to look at our work, but now we are in the domain of the second major challenge, the curse of expertise.

The Curse of Expertise

This term was made popular by the Heath brothers in their book “Made to Stick“. The concept is simply that the more you know about something, the more unique references, layers or knowledge and reference points you will have that will make it more difficult to communicate your ideas to people who are not experts. I offer up the image or these Star Wars fans and ask you how effective they would most likely be if they needed to explain the Star Wars movies to someone who had never seen them. Chances are they know so much about both trilogies, the original releases, the remade versions, the coming 3d versions, and the numerous games and comics that further matured the stories, it would be difficult for them to convey all of this in a simple thought. Likewise within our organizations, those people who know the business domain best will have the most difficulty communicating their understanding to a team of technologist who will experience the same problem when they try to explain the nuance of different possible technical solutions. Much like the Tower of Babel, an inability to speak the same language can doom the entire enterprise. It is in these situations where we can’t establish a clear shared vision, that people will refer back to their own experience, which brings us to the third challenge, functional fixedness.

Functional Fixedness

Human beings are cognitively to repeat pattern recognition once they make an association. This can be very useful when we have to make quick decisions, as it allows us to identify what’s around us and make rapid decisions. However, when confronted with creative problem solving, it closes numerous doors. The impact of this effect is that once you perceive something within a certain frame of reference, it will be very difficult to see it in another one. This was famously demonstrated by Karl Duncker with the candle problem. In this case, groups were given a candle, a box of tacks, and a match book. They were instructed to attach the candle to a wall so that it did not drip on the table below. The solution is to mount the candle within the tack box to the wall. What the experiment revealed was that the context in which the box was presented had a significant impact on people’s ability to solve the problem. Giving the participants the box as a discrete object, as opposed to simply the holder of the tacks, doubled the success rate of people trying to solve the problem (Dunker, 1945).

This is how we get into the situation where when you conduct user interviews you may find that the users can’t distinguish between their business operations and the actual system. It is the poor person trying to write a user story asking for the interface they have on their current system. This is not to begrudge them, they are acting predictably irrational. Rather it is the job of an analyst to help these people break from their context and reinterpret their problem in such a way as to better solve it. The idea that there is only one way to view a tool or process brings us to the last challenge I would like to highlight, the static solution.

The Static Solution

Western education teaches us that most problems have a single, best answer. Our simple task is to navigate the challenge, solve the puzzle and determine what the “answer” is. This is dangerous for two reasons, first it may cause us to think that our first answer is the answer and move on. Or it can cut the other way, where we become convinced there is a perfect answer and we can’t do anything until we figure it out. This dynamic is very real. When Tom Wujec conducted studies of participants playing the “marshmallow challenge” – a game where participants have 18 minutes to build a free standing structure out of paste, tape and string to hold a marshmallow as high as possible – he found that one of the groups which consistently performed worst was recent MBA graduates. Conversely, recent graduates of Kindergarden consistently ranked in the top levels of performance (Wujek, 2010).

Thus, armed with our conventional thinking, we seek to break apart and solve each individual problem. In fact, this is even where the word “analysis” comes from; it derives from the root meaning “to break into constituent parts”. While some problems are indeed closed in nature and capable of being solved in this dynamic, many real business problems are not. There is no “best” way to sell books, there is probably no “best” way to even sell books online. Even if there was one right answer, we exist in a world of changing circumstances, and the correct answer today, may be wrong in the future. In order to succeed in these domains of high uncertainty, we need to adopt a mindset that is much more comfortable with uncertainty. Break through innovative teams in numerous industries confront this challenge using techniques like emergent design, where a simple solution is built and then matured, or set based design, where multiple options are developed simultaneously as a means to explore the domain and develop the best possible solution.

So What is it an Analyst Does?

In an Agile environment, a good analyst helps craft an environment to confront these challenges. They need to be aware of how humans think and work to implement counter strategies to the tricks we will play on ourselves. The job of the analyst becomes one of helping people find their bearings so that they can remain heading towards whatever the true objective is. The key question they need to ask is, “just how are we going to get feedback?”

Agile projects talk about periodic demonstrations and frequent releases to production. These are good first steps, but probably are not enough. Demos in conference rooms are very constrained and many product can not be launched early in their development phases. Perhaps its bringing a version of the product to the line where will be used. Perhaps it’s usability testing or a gradual roll out strategy. These are critical questions and challenges where an analyst needs to be involved. Quite simply, the goal of a good analyst in an Agile environment should be to get the various perspectives of stakeholders and help them refine it into a single common view that is focused on valuable objectives. Good analysts will distill this by talking to all the different people, great analysts will do it by bringing those people together and creating an environment in which they can do it on their own. Once they have done that, they can dutifully play secretary and write down what everyone agrees upon.