How to have more insights

As presented in the previous post, cognitive psychologist Gary Klein proposed the triple path model to explain how we have insights. An insight, according to Klein, is an unexpected shift to a better story: a discontinuous discovery that gives us a more coherent explanation for the world around us. An insight is that lightbulb moment that enhances our understanding.

Of course we have to keep in mind that Klein’s insight is not the same as Johnson’s good idea. Insights are the wellspring of novelty, whereas good ideas –if implemented– are genuine innovations. Good ideas need insights, and conditions for the success of one should promote the other.

Let’s find out.

Hindrances to insights

And let’s begin with Gary Klein’s observations on the human obstacles to having insights. He identified the following four:

• Flawed beliefs – The unsuccessful twins relied on established theories and well-known data; they confidently rejected new evidence that didn’t fit their believes or convictions. By contrast, the twins having the insight employed more flexible strategies; faced with new or additional data, they were willing to explore new hypotheses and test them.
• Lack of experience – Klein found that it takes a good level of experience in the concrete situation or scenario to gain insight. Beyond mere knowledge, ‘an intuition‘ for the problem was essential for the successful twins; they were sensitised for the potential implications of emerging patterns that the less experienced twins missed.
• A passive stance – Just doing your job is not good enough to have insights. The successful twins were more active and maintained an interest in the wider context of their work. They were also more persistant in their efforts. By contrast, those twins who ‘stayed inside their assigned box‘ didn’t arrive at the crucial insight.
• A concrete reasoning style – Some individuals prefer the concrete over the abstract, the tangible ‘here and now‘ over the uncertain ‘then and there‘. They tend to consider a more exploratory approach as inefficient and easily loose patience. However, as Klein found out, such emphasis on the concrete and tangible reduces the opportunities for insights.

All of these very human perspectives share a common notion: they emphasise the concrete and specific; they enjoy the well-defined and certain; they prefer the predictable and safe; they are essentially static and inward-looking. In stark contrast, Johnson’s findings suggest a dynamic and outward-looking perspective: embracing the uncertain, accepting imperfection, and not getting trapped in trying to plan the unpredictable.

Where Klein found the human obstacles in our desire to insulate ourselves from the unknown and the uncertain, Johnson suggests that we should be comfortable with the unstructured and opaque situations in the fringes of our well-established understanding: that’s the adjacent possible that is the origin of good ideas. This rational will become even more evident when we have a look at the next set of obstacles Klein identified.

So let’s move to the technological obstacles to having insight. Believe it or not, Klein found the root cause for these obstacles in the design criteria for the very information technology that is meant to support us.

He focused on the most common guidelines for decision support systems. These IT-based systems should be designed to help us do our job better, they should display critical clues and filter irrelevant data, and they should help monitor progress toward defined goals.

For good reason, Klein summarised those four under the simple heading ‘Dumb by design‘. These guidelines are all predicated on the certainty that ‘the right approach‘ is entirely predetermined, which leads to numerous contestable assumptions: a well-defined task description; objective criteria for what is critical or irrelevant; clearly defined goals; and all of that should be immutable. This philosophy is then coded into a decision support system that will rigidly keep you on the defined track and even bring you back in line if you ever were to deviate.

In well-defined and stable situations, this logic is compelling; such systems are well-suited for optimising the efficient execution of standard processes and repetitive tasks. However, they are a recipy for disaster in unknown situations. The stronger the implementation of those design guidelines, the weaker and fewer the insights of the human customers.

The underlying assumption of entirely predetermined processes is grossly flawed and stands in direct opposition to Steven Johnson’s observations. Exaptation clearly depends on our willingness to look left and right, on our ability to think beyond narrowly defined initial goals. Serendipity guides us towards open networks and flexible arrangements to encourage unplanned encounters. And error means nothing but accepting imperfection; it reminds us that perfection is the wrong objective in complex and uncertain environments. For after all, those are the usual suspects that force us to be innovative, those are the situations when we need insights the most.

Moving on to the organisational obstacles, Klein specifically addressed the motivations that drive organisations: they are particularly allergic, and downright hostile, to surprise and error.

Organisations are relentless in their zeal to eradicate both. Yet going to extremes, they effectively undermine insights and end up in the predictability trap and / or the perfection trap. Insights come as surprise, they are unpredictable by definition; hence they disrupt the organisations’ quest for certainty. And unlike insights, errors can be defined, measured and dealt with fairly easily. According to such organisational logic, insights are an aberration from normal business, they disturb the standard procedure, they challenge the status-quo.

The reason isn’t simple, but rather elementary. Organisations are driven towards optimal allocation of resources and maximum efficiency. Which is code for an underlying ‘no-waste logic‘: no material, no energy, no money, no time, and no attention shall be wasted on anything that is not directly related to the given objective. 100% dedication to only the defined task.

This organisational logic presents enormous obstacles to insights; and the opposition to Steven Johnson’s observations is evident. Good ideas flourish in liquid networks (not too loose, not too rigid) which support serendipity and accommodate a little error. These conditions are essential to trigger unexpected encounters (of people and of ideas) that can give rise to new ideas, and to avoid that unconventional ideas –initially considered wild or strange, but potentially useful in the futre– are eliminated automatically and prematurely. Johnson advocates for ‘a little slack’ – certainly not as an end in itself, but as a kind of ‘allowance’, a tolerated flexibility, a domain where the usual controls are relaxed and where deviation from the norm is appreciated as the seed of something that could mature to become useful. That’s how insights can become good ideas.

So it seems that organisations and technology conspire with our most human anxieties to obstruct insights and to let good ideas wither. None of that follows an evil master plan. But all of it calls for our conscious and continued effort to do better.

Encouraging more insights

In the closing chapters of Seeing What Others Don’t, Gary Klein offers his suggestions on how we could have more insights. But rather than focusing on the obstacles described above (what we might call the enemies of insight) in order to fight and overcome those, he takes the positive approach the look for the friends of insight that can support our cause: the methods we can employ individually and in our organisations to have more insights.

Clearly, we have to help ourselves. And we can do so in several ways:

• Cultivate your Tilt!-reflex. That is your intuitive reaction that a new observation does not fit with your old model, saying ‘This can’t be right!‘ Most often, you will routinely reject such anomalies in your established thought pattern as errors. However, confusing as they may be, such contradictions and inconsistencies can be the origin of a new idea. Hence you shouldn’t discard them prematurely. This suggestion of Klein’s fits very well with Steven Johnson’s observations on error.
• Klein advocates swirl. Have lots of ideas floating around (in the air, on your desk, and in your mind) and let them interact freely to make accidental encounters and –potentially– new and useful connections. The connection path to insight thrives on such turbulent and unplanned interactions of previously unrelated ideas or observations. Here, Klein cites Johnson’s work explicitly, and he fully agrees with the need to foster serendipity.
• You should pause every now and then. Take a break, and a walk, to let the insight come to you. You cannot force insights directly through hard, focused thinking. But you can give your unconscious mind a chance to get to work, e.g., by reading the outline for your next article before going to bed, or by reviewing your notes from the last discussion before walking the dog. This emphasis on avoiding undue time pressures is similar to Johnson’s ideas on the slow hunch.
• Finally, critical thinking. For the creative desperation path in particular, when you are cornered, trapped, and under pressure, it is essential to critically review evidence and assumptions in order to spot and discard the weak anchor, to rebuild your story without it, and to arrive at that lightbulb moment: the insight that lets you escape. [Note: This specific advice would be counterproductive for the contradiction path and the connection path to insight.]

And we have to help our organisations as well to become more open to insights, to encourage them instead of fighting them.

Gary Klein had started his naturalistic investigation on insights driven by his interest in human decision-making. For him, increasing our performance in decision-making is the sum of two activities: reducing errors (represented as the ‘down arrow‘ in his presentations) and increasing insights (that’s his ‘up arrow‘). Through his research, he came to realise that our zealous efforts to reduce errors often derail our attempts to increase insights. We over-emphasise the down arrow at the expense of the up arrow. This paradox is particulary pronounced inside our organisations.

Therefore, Klein explains, helping organisations to have more insights means “breaking the tyranny of the down arrow“. Organisations must go beyond mere reduction of errors, they go beyond the narrow no-waste logic. They must learn to allow for a bit of surprise and cultivate a positive attitude towards some level of deviation from the established order. Ultimately, because we are our organisations, this is not so much a call to organisations to change themselves as it is call to ourselves to change our organizations: We have to shape them to be more open to insights, to foster them, to explore and exploit them.

This takes leadership! We have to set the right objectives for our organisations. Granted, efficiency is the one widely established management objective. However, efficiency is the perfect incarnation of the no-waste logic, this objective embodies the down arrow, and it is easily managed. But if that was to be an organisation’s only objective, then any future change in customer taste and interest, in market composition, in regulations, or in technology will likely kill the underlying business model, and the entire organisation with it [see Efficiency will kill you slowly].

Hence we must realise that innovation, agility and resilience are beyond the conventional horizon of efficiency as a management objective. For an organization to be capable of responding to future, uncertain and unpredictable challenges, it will depend on having insights and good ideas. Therefore, we have to give it a wider set of objectives and lead the organisation to embrace and implement them. This means taking off the blinders and loosening the filters to acknowledge the challenges that cannot be dealt with by just increasing efficiency. Of course that doesn’t suggest to abolish efficiency altogether, but it means working with more input from more and more diverse sources, and doing so in less perfectly structured way. As further guidance, Steven Johnson’s suggestions (above) provide a particularly apt characterisation of the circumstances we should seek to create in our organisations.

That said, we must keep in mind that not all organisations are created equal, and neither are the problems they’ll face. Hence we’ll need to cultivate an adaptable leadership style in order to foster insights and to promote innovation. I’ll take a deeper dive in an upcoming post.