Challenge
SiriusCyb, Inc., a young high-tech manufacturing company, was approaching a critical juncture. The company had officially begun production less than one year earlier and, while production had been a little slower than forecast, the board and the young CEO were happy with their progress. According to their original plan, SiriusCyb would outgrow their current manufacturing facility near the end of year two and would need to lease a new location. This was considered to be a major hurdle because the type and size of site they required was difficult to find and only occasionally came onto the market. Finding a new site, just when they needed it, was going to be difficult to say the least. Just the day before, their realtor had notified them that another company would be relocating and their facility might be available. But, if they wanted it, they needed to act quickly.
So, the CEO and his board had to make a decision that very well might determine the success or failure of the company. They could commit to the new lease much earlier than they had planned and guarantee that they could continue to fulfill customer orders in the future. However, if they took this route, they would have to pay to break their current lease and bear the expense of moving their machinery. They would also risk a loss of confidence from their current clients since the move would disrupt their current production run. More importantly, the lease on the new facility would be much more expensive than they had budgeted for. After much discussion, they concluded the move would only be feasible if actual production demand remained close to the original forecast. A significant shortfall would deplete cash reserves and force another capital raise — this time because of a bad decision on their part. The conclusion was that they had to have an accurate production forecast. But how? This was a new company with a new technique in a new market. The data required to do a traditional analysis did not exist.
Approach
SiriusCyb faced a very common challenge. Despite the proliferation of data, many major decisions arise in novel contexts where relevant information is scarce or non-existent. In this case, we were able to assist the CEO and his board by first acknowledging that their forecast was likely the best that would be available. If we accepted that, then we could reframe the question from "How to produce a better forecast?" to "How confident would we need to be in this forecast to act on the available lease?"
And, in order to answer the second question, we needed to consider possible options for meeting demand if we did not relocate now but the current forecast proved to be prescient. After some discussion, it was decided that there were several options for squeezing a few additional months of capacity out of their current facility. While this would inflict some pain and could significantly impact their financial performance, it would not prove to be disastrous. Their current forecast called for a production volume of 1150 units by month 20. SiriusCyb collectively decided that they would need to be at least 80% confident that they would be producing a minimum of 850 units by month 20 in order to proceed with the current lease.
At that point, the problem became: "How to quantify confidence?"
Solution
Our first step was to identify potential sources of useful data. We identified three:
- The CEO who had studied the market and established the forecast based on his research
- The board members' collective entrepreneurial experience and knowledge
- Three high-tech manufacturing companies who were similar to SiriusCyb (even though in distinctly different markets) and who had been established several years earlier
To quantify the CEO and board members' confidence in the original forecast at month 20, we individually walked them through an elicitation process that translated their intuition about best-case, worst-case, and most-likely outcomes into quantitative probability distributions. After some refinement, each member agreed that the distribution reflected their outlook. We then built a Bayesian model that used these distributions as the prior belief about possible production outcomes.
As evidence for updating the prior, we used the forecast accuracy of three comparable companies. Our interest was not their actual production at month 20 but in how that production compared to their early forecast. In other words, we were trying to answer the question: "How good are early stage high-tech manufacturing CEOs at forecasting?" This is essentially what Daniel Kahneman famously called the Outside View. For our data points, we used the percent of their forecast they attained in month 20.
Our model produced the following posterior distribution and calculated that the probability of reaching at least 850 units by month 20 was approximately 45%.
After showing the model output we made sure that the board understood that, as with any probability calculation, interpretation is key. In this case, the small data set used for a prior made the model very sensitive to even small changes in the inputs. However, even if we were to multiply the output by a factor of two, we would be just a little above the minimum confidence level of producing the minimum amount needed to make the lease viable. In addition, we explained that the model made no statement about the accuracy of the forecast. The forecast could very well prove to be extremely accurate. This exercise was designed to provide a binary answer to the question: "At this time, and given the alternatives, do we have enough confidence in the forecast to risk an early commitment to a new facility?" And, that answer was, quantifiably, "No."
SiriusCyb is a fictional company created to illustrate how Inlytica applies Bayesian methods to business decisions. Details are adapted from real-world cases, but identifying information has been changed.
← Back to Writing