Since the rise of the “analytics revolution” in the 1990’s, intuition and judgment have gotten a very bad reputation. Making decisions based on experience, we were told, is just too fallible. Humans are notoriously biased and irrational. In modern organizations, judgment would be replaced by statistics and decisions would be based solely on patterns and trends extracted from a vast pool of objective, unbiased data.
It all sounded great but it never really happened. And, now, there seems to be a grudging willingness to admit that maybe this vision was based on a misplaced faith in data. In fact, there is mounting evidence that basing a decision process solely on analytics can be as problematic as relying only on judgment.
Here’s just three reasons why:
Most organizations do not have as much data as they think
The reality is that very few companies actually have so called “big data”. Even if an organization possesses terabytes, that store will likely represent only a very small fraction of the potential data universe. And, that fraction will get even smaller after cleaning and filtering. Analytics based on small sample sizes can lead to large errors in estimation and unreliable conclusions.
Data contains patterns – Analysts will find them
At its core, analysis is an attempt to uncover patterns and trends that exist in a data set. The problem is that patterns are often too easy to identify and statistical methods such as p-value can exaggerate their significance. Frequently, what seems to be a clear alignment of data actually has no meaning whatsoever. Even if an arrangement of data is determined to be meaningful, analysts often mistake correlation for causation. And, even if causation is established, a pattern is predictive only to the extent that the future will resemble the past. Choosing a course of action based on faulty insights can result from data driven decision making as easily as from a judgment based process.
Data itself can be biased
Information extracted from data is often itself biased. Analytics is a human endeavor and biases can result from the methods used to collect data, the sources it was collected from, and the filtering and cleaning process. Even the type of data and the way it is stored can create biases because it may influence whether or not it is included in a decision making process. Instead of being free from preconceived opinions, decisions made with data can actually contain additional biases introduced from the analytical process.
So, what is the point? The point is that for too long now software vendors and business consultants have exaggerated the infallibility of data when it comes to making strategic decisions. Data is, of course, extremely important. But, it should not replace judgment and intuition. The decision process should almost always incorporate both. One of the most important steps in making a decision will be determining how much influence one should have on the other.
To find out how we can help contact us.