Intuition seems to be back in style. After decades of "data-driven decision making," academics and consultants are now acknowledging that human experience and judgment may have a place — possibly even a vital place — in the decision making process. Lately I've seen article after article arguing that the best outcomes come from a mix of data and intuition. That's great and, I think, absolutely true. But, it leads to another question that has yet to be answered: How? How do you actually go about combining analytics and experience when making high stakes decisions?
Think about this scenario:
You are the CEO of a consumer packaged goods company, and you have to decide whether or not to launch a new product line into the European market. First, you and your executive team would likely review whatever data your analysts can produce: Nielsen/IRI sales trends, survey data, and pricing benchmarks. What you all see seems to suggest growth but, as is usually the case, it is inconclusive. Data exists for some countries but not for others. For some countries you have decades of data but for others only a few quarters. And, the products that are studied are not an exact fit for what you have in mind.
Next, you would probably listen to experts. You would get presentations from Regional sales leaders and product managers, and maybe a consultant or two. Some say consumer interest is surging; others warn about cultural taste preferences and entrenched competitors.
Finally, you might gather the conflicting opinions of your team who all have varying levels of experience in these situations.
Now what?
This is the real world. Most high stakes decisions are made in environments where data is sparse and noisy and experts disagree. The articles that I have read suggest that this is the time to "listen to your gut" and make the call. But, you have to have some justification for your decision when you explain it to your board. What if they ask how confident you are, why, and what evidence you relied on? What if the decision goes badly and you're challenged months later to show your process? The trouble with the "now just rely on your gut" theory is that it doesn't look that different from guesswork.
So, what do you do?
I suggest that a vital step in solving this problem is to stop thinking in terms of Intuition AND data and to start considering Intuition AS data.
What many people don't realize is that your brain stores patterns and associations — essentially memories of past interactions, successes, and failures. It holds on most tightly to those marked by strong emotion, repetition, or vividness. But, instead of keeping a database of these experiences, it compresses them into mental "models" of the world — a kind of rules-of-thumb playbook — so you can quickly anticipate future events. It may not look like a SQL table, but it's still data that can be extracted and used. The problem is, we need different "ETL tools" to extract, transform, and load intuitive insights into a form that can work with quantitative data.
The good news is that these tools already exist. In fields like engineering, risk analysis, and public policy, researchers have developed structured methods for capturing expert judgment — a process called expert elicitation. These techniques give us a disciplined way to extract what people know (or think they know), transform it into probabilities, and then load it into a framework where it can be combined with whatever hard data we have. Once the intuition has been "translated" into data, we can use methods like Bayesian inference to merge it with quantitative evidence.
You can't predict the future with certainty, but you can build a rigorous, defensible process for making the tough calls that shape it.
← Back to Writing