Whether people realise it or not, "freedom to choose" is the underlying principle behind many of the agile practices. We call this principle Real Options. An understanding of Real Options allows us to develop and refine new agile practices and take agile into directions it hasn't gone before. Real Options also help us understand why some people resist some of the practices.
To use a familiar phrase, Real Options is about "deferring decisions to the last responsible moment," which is an explicit principle in the Lean Software approach. By avoiding early commitments, you gain flexibility in the choices you have later. A more formal definition will follow.
Viewing Agile through Real Options
Real Options are everywhere in agile practices. To illustrate this let's look at some agile practices and point out where commitments are avoided as long as possible. These are just a few examples. Once you become aware of Real Options you will discover a lot more.
Behaviour Driven Development and Test Driven Development provide many options, especially "the option to change the software, knowing when you have broken it." Without test driven development, there would be no refactoring. In other words, test driven development removes the need for design commitments, the choice of design can be deferred until after the coding has started. More subtly, Test Driven Development allows the developer to know with certainty that they have finished, instead of determining beforehand when the programming is done. Test driven development requires no decision at all, simply stop coding when all the tests are green. Similarly mock objects allow the developer to defer the decision on how to implement a class until a later time when they have more information on what is required from the class. They also create a nice recursive implementation pattern.
XP and Scrum defer the decision about which story to develop until just before the coding starts. This allows the team to incorporate information that arrives at the last moment, such as a new client request. The Scrum Backlog provides a forum where any idea for functionality can be recorded without requiring an immediate commitment to build it.
At the project Chris is working on, the development is prioritised based on client requests. In effect, sales drive the prioritisation process by stating the relative importance of a client. The team doesn't start working on a feature until the clients have requested it. "Strategy" meetings were cancelled several months ago because they were just crystal ball gazing about the features the team thought the clients liked. Now the team waits to see what the clients actually request.
By delaying commitment on the features built, the team is able to reduce time-to-market for new features requested. When the client requests a feature the team is free to act, because they are not tied up working on unwanted features. This is only possible with short iterations and a fast turnaround on regression testing.
Pairing provides options in that more than one developer will have an intimate knowledge of any part of the system. This mitigates the "truck number" (a.k.a. "lottery number") risk or "key man" dependency. (The truck number is the size of the smallest set of people in a project such that, if all of them get hit by a truck, the project would be in serious trouble [1]. By spreading the knowledge of the system the project is not in danger when anyone on the team wins the lottery.)
So What are Real Options?
Real Options is an approach that allows people to make optimal decisions within their current context. This may sound difficult, but in essence it is a different view on how we deal with making decisions. There are two aspects to Real Options, the mathematics and the psychology. The mathematics of Real Options, which is based on Financial Option Theory, provide us an optimal decision process [2]. The psychology of uncertainty and decision making (based on Neuro Linguistic Programming [3][4] and Cognitive Behavioural theory) tells us why people do not follow this optimal decision process and make irrational decisions as a result.
Financial options are options written and stock and shares. A huge amount of mathematics has been developed to price and risk manage these options. "Real Options" is the term we've coined to describe our use of financial options mathematics to help us make optimal real world decisions... Real Options are real world decisions.
Much of the existing literature explains how we can use the Black Scholes formula to price real options. Fisher Black and Myron Scholes along with Robert Merton are Nobel Prize winning economists who invented an equation to value financial options. Unfortunately, most of that literature is wrong when applied to Real Options. We cannot price real options using Black Scholes' famous equation. Whereas a PhD or Masters in financial mathematics is required to derive Black Scholes or prove the statement we just made, no understanding of maths is required to use the results of these maths.
The "only" things that the Financial Option maths can tell us about options are:
- Options have value.
- Options expire.
- Never commit early unless you know why.
Quite simply, the maths tells us not to make a commitment until the last minute unless we know with certainty why we want to make the decision early. Sound familiar?
The "Real Option" decision process is not new. Preston Smith was one of the first to coin the phrase "Make the commitment at the last responsible moment" [5]. However, whereas Preston's statement was an opinion based upon many observations, Real Options are based on financial mathematics.
In the Summer of 2006, we ran a series of sessions on Real Options. One participant, Iain Brocklehurst, a senior IT manager within JP Morgan said "This is just bottled common sense." Real Options, while based on maths, focuses on the psychological aspects of decision making which are probably more important than the maths. Without understanding the psychology, most attempts to implement a new decision process will fail.
Choices
Given any decision to be made, there are three possible decision categories, namely, a "right decision", a "wrong decision" and "no decision". Most people think there are only two; you're either right or you're wrong. As we normally do not know which is the right or wrong decision, the optimal decision is in fact "no decision" as this defers the commitment until we have more information to make a better informed decision.
However, if we observe the behaviour of most people, we notice that an aversion to uncertainty means people make decisions early. Real Options address this aversion to uncertainty by defining the exact date or conditions to be met before the decision should be made. Instead of saying "not yet", the Real Option approach says "Make the decision when.....". This gives people certainty over when the decision is made and as a result they are more comfortable with delaying the decision. Delaying commitments gives decision makers more flexibility as they continue to have options. It allows them to manage risk / uncertainty using their options.
Real Options feels unnatural to begin with. Like leaning down the hill when skiing, or leaning back when you jump from a height on horseback. A few people adopt them naturally. Others need a coach, time and practice to get it. Sound familiar? Perhaps like Agile? Perhaps the "unnatural" feel of this uncertainty one of the reasons that some people are so opposed to Agile.
Rather than decisions, Real Options is really about commitments. Quite often people make a decision which becomes an emotional commitment to an idea. They do not realise that they can change their minds.
Is agreeing to go to out with a friend an option or a commitment? Most will say a commitment because they have an agreement with their friend, while in reality it is an option. If something happens that provides you a higher value (like your dream partner asks you to go backstage at a U2 concert with them), you'll call your friend to reschedule. An agreement is an option.
Financial options have a contractually defined expiry date (date on which they end, or expire). The expiry date of Real Options is conditional rather than contractual. The most powerful aspect of Real Options is the ability to push back the expiry date of an option, thus giving more time for an optimal solution to be discovered. There is value in paying to defer a commitment.
Examples in business
These practices are already in use. As said before, this is nothing new. Three well known companies are already using this successfully: Toyota, Microsoft and Google.
Toyota's relationship with options is well documented in Mary and Tom Poppendieck's "Lean Software Development" [6]. Famously, Toyota waits until a customer buys a car before they start to build it. How's that for deferring commitment?
Microsoft's relationship with Real Options is well known, like the famous trade show where the Microsoft stand "looked like a bazaar". While other companies were betting their company on a single product or strategy, Microsoft was hedging its position so that it could still win the office automation battle even if it lost on the operating system front. Microsoft is the ultimate master of the wait, and wait, and wait, and see strategy. Consider the Internet Explorer episode where Microsoft waited until the internet had emerged as a technology before moving in. A question we would love to have answered is: "How many programming teams and solutions did Microsoft consider when faced with this crisis?" Did it adopt the traditional "cost optimising" best solution approach or did it look at many alternatives? We suspect the answer is fairly obvious.
Google is one of the first Informationalist [7] companies. Rather than enter into conflict with its customers, it cooperates with them. No random banner ads, but rather adverts for products and services that you might actually be interested in.
The Future of Agile
It is often too easy to destroy someone else's options. As a result, Real Options works most effectively if participants cooperate with each other. This goes against common practice of hiding our intent. However, we can learn from Google's approach. After all, how can I help you if I do not know what you want to achieve... Perhaps this should be starting point of Agile management practices.
We need to learn how to listen so that we can help other people achieve their goals. Ironically enough, "Listen" is one word that does not appear in the APLN's "Declaration of Inter-dependence". Leaders must provide and create options to for people they work with instead of making decisions for them.
Real Options is a set of principles that validates certain practices, as shown above, and can be used to generate new practices. Currently agile needs to develop Leadership and Management practices (beyond project management) that support the agile mindset, to move out of the IT department and into the rest of the corporation.
To give a few examples of how this would work, let's look at decision processes and resource management.
Decision process
The optimal decision process is as follows:
- For each decision, identify the options available.
- Identify the last point at which a decision can be made. i.e. the conditions to be met to make a commitment. This is calculated by working out the deadline for the implementation of an option, and then working back to the decision point (i.e. the Takt time). The first decision is made when the first option expires.
- Until that expiry date, continue to look for new options.
- Attempt to push back the decision time. Quite often this is free or very low cost. To do this, we need to be able to implement the option as quickly as possible. During slack time, work on how to speed up the process (just like Toyota does in their production plant).
- Understand that cost optimisation is not the same as revenue optimisation or risk reduction. Sometimes it is worth investing in more than one option even though this may cost slightly more. After all, options have value (just ask Microsoft).
- Wait to make decisions... and wait... and wait... until the conditions are correct.
- When you need to make a commitment and act... do so as quickly as possible. And you can proceed with confidence because you know you have made the best informed decision possible.
Recently I was in a meeting where we were discussing the need for a "Plan B". We had a "Plan A". There was an argument about the need for a "Plan B" at all. The success of "Plan A" was quoted as 99%, 50%... and many other percent. I applied Real Options and suggested we first identify when to cut across to "Plan B". We also identified the additional up-front investment required to make "Plan B" an option. It transpired that about ten minutes of effort was required but it would take a few days to do because of the large number of steps and departments involved. We set the "Plan B" option in motion. In addition, we sketched out "Plan B" in rough detail. "Plan A" failed. We knew exactly the time to swap over to "Plan B", and everyone knew what they had to do. The switch over was a smooth process. Far from the emergency expedition of tasks, and resulting chaos, "Plan B" went so smoothly that many people did not even realise we had changed course at the last minute. -- Chris Matts
One of the key lessons to take from Real Options is that the right time to decide whether an idea is a good one is after you have tried it, rather than before you have listened to it.
Resource Management
Real Options, when applied to resource management, tells you that you should deploy resources from the least experienced up. Your most experienced team members with the most options (most skills etc.) should be allocated last. This means your most experienced staff can be deployed to address any emergent issues. While unallocated, they can coach junior members (create options) by pairing with them. This ensures they can ensure good practice and keep track of the architecture of the application. This could be the PM or lead developer. They should learn to be lazy - a good start is to read Tom DeMarco's "Slack" [8].
Executives in organisations should adopt an option based investment strategy rather than committing to project budgets. The executives should defer commitment to invest the full amount until they see the success of initial investment or validation of a business case. The investment should be made for each each release based on success criteria. The success should not relate to the development but rather the business project. i.e. the increase in business value. A project where the development of a feature costs double the original estimate would be a success if the revenue increase was high enough. Similarly, the perfect project that costs half the estimate should be canned if it does not generate enough revenue. Executives should be able to increase the investment in a project if it is going well. In order to do this, they need liquidity of development staff which is facilitated by Real Option based resource management.
Finally, executives in an IT-related company should consider adopting customer-driven development where real customers choose and prioritize the features (i.e. customers which are external to company, not the XP kind which are part of the project team), . In order to achieve this, the Sales and IT Development should be integrated. Sales gives an a la carte menu of features (options) to the customers rather than a feature list for the next release. Sales offers anything on the menu, or the option to chose something that's off the menu. To introduce a new item, the team brainstorms the features, does just enough analysis to determine the Takt Time for the feature, and identifies the major risks. The Sales team is given these Takt time estimates so that they can offer them to customers. If the customers don't want them, don't build them. If they do, do. In this way you avoid building the 67% of features identified in the Standish Chaos report as "Rarely or Never Used"[9].
This is the approach we are taking on our current project. Sales loves it. Instead of selling the customer features they do not want, they ask them what they want and give it to them. Sound familiar? (If it works for Google...) We've seen the sales team start to invite the project team to client meetings so that we can explain the approach in more detail.
There are many practices out there to be discovered...
A word of caution. Real Options is an active risk management strategy. It is necessary to constantly monitor your portfolio of options to make sure they are not being destroyed. In addition, it is an information hungry process as it requires the practitioner to actively look for information. Also remember that doing nothing is an option. To do nothing must be part of an active decision process. Real Options is not an excuse for procrastination.
Conclusion
Real Options is the underlying principle of agile. Many of the agile practices defer decisions as long as possible. Succesful companies like Toyota and Google are using Real Options to their benefit. Real Options provides us with the ability to take the agile mindset and project it into areas so far untouched.
As a final thought: last year Chris met with a friend in a London pub. Unlike many people, she instantly "got" Real Options. In the next ten minutes, she worked through many of the implications of Real Options. (It has taken us six months to work out what she had worked out in ten minutes). Then she fell silent. After a few minutes of silence, she said "They are about creating a better work place." She's right, but there is more. That's why we do them. Not because they will make us rich, which they will, but because they are fun.
Have fun. :-)
References
1. http://c2.com/cgi/wiki?TruckNumberFixed
2. http://en.wikipedia.org/wiki/Real_option
3. "NLP Unplugged", Gary Sage & David Hagan,
4. "User Manual for the Brain,' Bodenhammer, Bob G. & Hall, L. Michael
5. "Flexible Product Development" Smith, Preston G.; (Sept 2007)
6. "Lean Software Development", Poppendieck, Tom and Mary; Addison-Wesley Professional (May 2003)
7. Informationalism: "a
technological paradigm based on the augmentation of the human capacity
of information processing and communication made possible by the
revolutions in microelectronics, software, and genetic engineering." From Castells, M. "Informationalism, Networks, and the Network Society: A Theoretical Blueprint." See also http://en.wikipedia.org/wiki/Netocracy.
8. "Slack", De Marco, Tom; Broadway (November 2001)
9. This chart is visible on page 5 of this document: www.clarkeching.com/files/why_your_business_needs_agile_software_development_v1.0.pdf
Rose photo credit - Ed Parcell
About the Authors
Chris Matts is a project manager and business analyst with over ten years experience developing trading and risk systems for Investment Banks including JP Morgan Chase, BNP Paribas and Dresdner Kleinwort Wasserstein. A former developer, Chris has a Master Degree in Mathematical Trading and Finance (including financial option maths) and a MEng in Micro-Electronics and Software Engineering. Chris is currently the programme manager on a product where he is using Real Options and Agile techniques to integrate the Sales and Projects teams.
Olav Maassen is chief engineer at QNH, the Netherlands and has nine years experience in IT doing projects for financial institutions. He is co-author of "Applied Java Patterns". His main interest is in helping teams develop software more effectively. Olav strives for continuous improvement both for himself as for those he works with.
Currently Chris and Olav are collaborating on a book about Real Options and how they really work.