Bill's Notebook

From an early age, I have never liked the observation that something is complex. It usually meant that person is just resigning themselves to never understanding the problem. I couldn’t stand this defeatist attitude and have spent most of my life trying to devise ways to tackle complex problems including the aptly-named “wicked problems.” Even though I may never find the solution to the P versus NP Problem, it has taught me a great deal about problem solving in general.

So, what do we mean when say a problem is complex? According to Dr. Melanie Mitchell, there are nine definitions for complex as used by complexity theorists. These definitions range from “complex as a matter of size” to “complex as a degree of hierarchy” to “complex as a measure of algorithmic information content” (pp. 96-111). I tend to think of complexity in terms of systems theory in which you have a number of discrete components with numerous feedback loops and many variables that are hidden within the system processes.

A good example of a complex system is the American economy. There are many discrete components in the forms of companies, consumers, banks, regulatory agencies, etc. all passing information to each other and reacting to that information. Attempts to model the American economy range from the simple macroeconomic diagrams in textbooks to detailed microeconomic equations that requires years of mathematical study to even understand. Yet these models, no matter how detailed, cannot fully describe and fully predict how the American economy operates.

If you accept my definition of complexity then you can see how the next concept describes why complex problems are hard to solve. We have difficulty in solving complex problems because our observation of the problem is hindered, we cannot fully understand the problem, our decision-making processes are flawed, or we cannot act appropriately in confronting the problem. If any of the difficulties I mentioned sound familiar it is because I am describing the four components of the “OODA Loop.”

The Observe-Orient-Decide-Act Loop (OODA) was created by Colonel John Boyd who was a fighter pilot and scholar in military strategy. This concept has been adopted both by the U.S. military and championed by such business experts as Tom Peters. As the diagram below demonstrates, a person, team, or an organization observes a situation along with other inputs. Based on the observations and several internal factors, the subject attempts to orient themselves or understand the unfolding situation. Based on that understanding, the subject then makes a decision and acts upon that decision. Throughout the OODA Loop, there are several feedback channels that make the entire process nonlinear.

Colonel Boyd explained that the use of the OODA Loop was to travel through the Loop faster than your opponent. You present confusing and ambiguous information to your opponent so that they have difficulty orienting themselves and thus are slower to decide and act. Essentially, you want to go through your own OODA Loop faster than your opponent does so that they start falling behind and then are paralyzed by their inability to analyze the situation. Time is the key factor in OODA Loops.

The OODA Loop is why I think complex problems are so difficult to solve. Consider the five components of the OODA Loop as it applies to your personal abilities or the abilities of your team/organization:

• Observe: This is the beginning of the Loop and also feeds into another iteration of the Loop. If your observational abilities are hindered or you just cannot observe all parts of the unfolding situation then you are working with incomplete information. History is replete with examples where disasters occurred because of the lack of key information.
• Orient: This is where you/the team/the organization takes in the new information and pairs it with your previous knowledge, cultural traditions, and other internal factors that influence how you process and analyze information. So, even if you are able to observe the entire unfolding situation, your internal abilities to process and analyze this information can prevent you from fully understanding what is happening.
• Decide: This relates to your ability to generate hypothesis about the situation and possible responses. There is the common “paralysis by analysis” which hinders decision making because you are still trying to orient yourself to the situation. Or, even if the organization has a good understanding of the situation, decision processes may be so cumbersome that you cannot make a decision in time to act on the situation.
• Act: You may not have the resources to act promptly and/or appropriately. Your understanding of the situation may have led to a flawed decision that forces an invalid response to the situation. You do not have the proper feedback mechanism built in your action to determine how your act affected the unfolding situation.
• Feedback: As you go through the OODA Loop, you are constantly generating and receiving feedback from your current iteration and previous iterations. Without good feedback design, your own actions can contribute to the ambiguity of the situation. This is especially true of wicked problems where there is no consensus on the actual shape of the problem and your actions can drastically morph the problem into a completely new problem.

The good news here is that you can also use the OODA Loop to better your abilities to handle complex problems. Use the five components as a checklist for improving your (or your organization’s) processes in handling complex problems.

For example: how well do you observe? How good is your organization at collecting and disseminating information internally? Do your people have the necessary prior knowledge and analysis skills to properly orient themselves when new observations come in? How robust and quick is your team’s decision-making skills? What barriers can you remove so that you can act faster? What can you do to improve your feedback mechanisms?

Government is going to face more complex problems especially in a climate of reduced budgets and increasing responsibilities. All government employees at all levels need to sharpen their problem-solving skills so that we are more innovative and can better tackle the looming wicked problems that face the nation. Whether you accept my suggestion to use the OODA Loop or come up with your own problem solving method, the process of thinking about complex problems is a great way to sharpen your problem solving skills.

Reference:
Mitchell, M. (2009). Complexity: A guided tour. New York: Oxford University Press.

It was in my second year of being a Presidential Management Intern when I was feeling rather cocky after a string of successful projects. So, when I met with my boss for our weekly status meeting, I was casually leaning back in my chair just radiating gloat. That is when he leaned forward and said, “you are only as good as your last project. What have you done for me lately?”

It was that advice that has guided me ever since. It is very easy in the euphoria surrounding the triumph of solving a difficult problem or pulling off the near-impossible project to not spend the time questioning just why you succeeded. To do so seems to be diminishing the success and even doubting that you actually did succeed. On the contrary, an objective review of how you succeeded will greatly help you in continuing to succeed.

When we succeed, we can become victims of three biases, according to Gino and Pisano (April 2011). There is the attribution bias in which we overestimate how our knowledge and actions contributed to the success and we downplay any external factors that could have just made us more fortunate. We also become overconfident in our abilities as we tackle the next challenge. The third bias (and which I believe is most important) is that we don’t ask why we succeeded because the success is proof enough.

To illustrate this last bias, Gino and Pisano (April 2011) recount a study in which students were given a set of math problems to complete. When the students submitted their answers, they were only told if they had the answer right or wrong. The students were given time to reflect before they were given a second set of math problems. The second set was designed so that a key concept in the first set of problems was needed to solve the second set. The students who successfully solved the first set of problems generally spent much less time reflecting before they started on the second set of problems. Thus, many of these students failed to find the answer for the second set of problems. Reflection, whether the student succeeded or not, is the key to continuing to be successful.

So, how do we best learn from success? We should celebrate success but also examine the causes of success. For every project, we should hold a systematic review. Gino and Pisano (April 2011) give the example of Pixar’s review process. Even though Pixar has had eleven hit animated films in a row, the company still goes through an exhaustive review process to determine what made the film successful and how to repeat that success.

Another point to keep in mind is to fully investigate the feedback. Was it immediate or at least can be connected to the actions taken? Is the feedback a true indicator of success or just a random event that looks like a successful outcome? Feedback is an important concept and I explore it in greater detail in this discussion posting.

Two final points. First, “[r]ecognize that replication is not learning” (Gino and Pisano, April 2011). Blindly following the same formula again and again can suddenly turn against us as the nature of the problem changes and what worked before doesn’t work now. And, second, we should always be experimenting. We can always improve how we do something. Plus, we can create variations on our actions that may not apply to the current situation but can apply to a challenge in the future.

Failure is a great teacher but so is success. Learning from our successes will keep us from becoming “one-hit wonders” and give us the string of successful “hits” to be “rock stars.”

Reference:
Pino, F., & Pisano, G.P. (April 2011). Why leaders don’t learn from success. Harvard Business Review. 68-74.

The three keys to innovation are to seek out new ideas, test these ideas on a scale where failure is survivable, and constantly monitor these trials for feedback. This is according to Tim Hartford’s new book, Adapt: Why Success Always Starts With Failure. Hartford argues that the world is too complex for top-down “big project” innovation based purely on expert judgment. The best path to innovation is to try a lot of ideas simultaneously (even if they contradict each other), build in robust feedback loops, and use the winning ideas to start a new round of trials.

This is not a new method of innovation; in fact it is the oldest method of innovation around – evolution. Nature is constantly creating variations of species and then selecting the species that best survive current conditions. What Hartford does is apply that concept to organizations to see if a similar process works in determining what companies succeed and which go out of business. The organizations that best survive a constantly changing business environment are the ones that combine incremental improvement along with the occasional long-shot idea to propel them into a better part of the business landscape ahead of their competitors

So, what does this have to do with government agencies? Hartford flatly states that this innovation method will not work in government agencies because of several barriers. First, there is not enough time for political appointees to fully see these experiments through before a new administration comes in office. Second, the process depends on a large number of failures for innovation but failure carries a high stigma in government. Third, it is difficult to clearly demonstrate that a policy innovation actually had an effect due to the lack of robust feedback loops in government.

I believe that Hartford’s opinion about government innovation is way overstated. There have been numerous government projects that have been extremely innovative: the Hoover Dam, rural electrification, the Interstate Highway System, the Moon landings, the Space Shuttle, the Internet, and so on. Even so, when you examine how these agencies developed these projects you do see that these agencies tried many ideas and learned from these trials. NASA has an amazing knowledge management culture and DARPA’s successful record of innovation is built on the concept of trying many long-shot ideas at once.

What holds government back from being even more innovative is the stigma of failure. Many agency cultures are too cautious because of the constant external scrutiny and the internal cultural practices of not sticking your neck out and just waiting out the latest change effort. In many cases, this caution is well-warranted. Many people depend on government agencies and thus agencies cannot fail in their primary mission of delivering Social Security checks, defending the nation, or enforcing laws and regulations

But failure to innovate will also lead to mission failure for agencies. In the sixth chapter of his book, Hartford describes how the 2008 economic meltdown was inevitable given the tight coupling of economic institutions and the failure of the government to prevent financial organizations from becoming too entangled. He argues that in any complex system, accidents will normally occur and that often our failure-prevention efforts will only increase the probability of failure. What is needed are the twin strategies of placing buffers between parts of the system and setting up feedback loops to warn us of emerging failure events.

Government has to constantly innovate so that it can continually deliver on its mission. This means that the culture has to change so that the agencies accept the small failures that teach to avoid the large failures that cripple the agency and harm the people it serves. Whether we call it “experimentation”, “pilot tests”, or some other euphemism, the better government is at innovation the better it can serve its citizens.

Disclaimer: The opinions in this posting are solely mine and do not reflect the views and opinions of my employers or any organizations I belong to.

References:

Belfiore, M. (2009). The Department of Mad Scientists: How DARPA Is Remaking Our World, from the Internet to Artificial Limbs. Smithsonian.

Hartford, T. (2011). Adapt: Why Success Always Starts with Failure. Farrar, Straus and Giroux.