Forecasting demand: when to ask experts, when to build a model
For a buyer or finance director who must forecast demand for a new product and for a line that has sold for years. With data, start from statistics; without it, use structured judgement such as Delphi. This post maps the choice between the two.
Director, KDIGI
Shop K (an invented example) sells bedding and has six years of monthly sales. This quarter the owner must fix the purchase for the next three months and, at the same time, decide the first order for a new range of duvets that nobody has sold yet. The two jobs call for different forecasting methods, and mixing them up is where many people lose money. This post draws a map from structured judgement to time-series decomposition, then gives a table for choosing by situation.
How do a forecast, a goal and a plan differ?
Hyndman and Athanasopoulos (2021) open their book by separating three things that are often run together: forecasts, goals and plans (Hyndman and Athanasopoulos, 2021). A forecast is the best estimate from the information available. A goal is what you want. A plan is what you do once you have seen both (Hyndman and Athanasopoulos, 2021). The mix-up shows clearly when salespeople forecast for themselves: if the number is also their target they have a reason to set it low so it is easy to beat, and if they are optimistic they set it high (Hyndman and Athanasopoulos, 2021). The authors advise keeping forecasters apart from the people who use the forecast.
The five steps in the book give a frame for every method below: define the problem, gather information, do a preliminary analysis, choose and fit a model, then use and evaluate it (Hyndman and Athanasopoulos, 2021). According to the book, the first step is often the hardest, because you must know what the forecast is for and who will use it.
Five-step forecasting process
A demand planner or finance director needs a sales forecast: in what order should the work go so that the result is usable for decisions?
- Five steps
- Problem definitionOften the hardest step: understand how the forecasts will be used, who needs them and where forecasting sits in the organisation. Talk to everyone who collects the data or uses the forecasts.
- Gathering informationAt least two kinds: statistical data, and the accumulated expertise of the people who collect and use it. If history is too short, use judgmental methods.
- Preliminary analysisAlways start by graphing the data: consistent patterns, trend, seasonality, business cycles, outliers that need explaining.
- Choosing and fitting modelsThe best model depends on the history available, the strength of relationships with explanatory variables and how the forecasts will be used. It is common to compare two or three.
- Using and evaluating the modelProduce the forecasts, then evaluate them. A model can only be properly judged once data for the forecast period are available.
No data yet: three structured ways to use judgement
The book names three settings for judgement: no data at all, data available with judgement used to adjust the statistical forecast, or statistical and judgemental forecasts made separately and then combined (Hyndman and Athanasopoulos, 2021). For Shop K's new duvets, the first setting applies. The three main methods:
| Method | What you do | Weakness according to the book |
|---|---|---|
| Delphi | Assemble 5 to 20 anonymous experts; each answers with reasons; a facilitator summarises and feeds back; repeat two or three rounds; combine the results | Slow; more rounds mean more drop-outs; the facilitator needs experience |
| Forecasting by analogy (structured form) | Find similar cases, forecast from each, rate how similar each is, optionally take a weighted average | Hard to find many analogies; a single analogy easily biases the result |
| Scenarios | Build several plausible situations, including extreme ones, then prepare responses | Each scenario may be unlikely |
According to Hyndman and Athanasopoulos, Olaf Helmer and Norman Dalkey of RAND invented Delphi in the 1950s, on the assumption that forecasts from a group are generally more accurate than those from individuals (Hyndman and Athanasopoulos, 2021). Anonymity is a key feature, because experts cannot be pressed by rank or by the loudest voice; the book's authors also note that group meetings tend to raise optimism (Hyndman and Athanasopoulos, 2021).
The book's example of analogy is worth remembering. A group of academics and teachers in Israel were each asked how long it would take to finish a school curriculum; answers ran from 18 to 30 months. One member, an expert in curriculum design, looked at similar projects around the world: 40% were never completed, and the rest took 7 to 10 years. The Israeli project took 8 (Hyndman and Athanasopoulos, 2021). The lesson for Shop K: before settling on a number for the new duvets, ask how similar ranges sold in their first month at other shops.
Two other methods are common: collecting forecasts from the sales staff and taking the opinion of the executive team. The book places both among the less structured methods and more prone to bias: salespeople forecast and are judged on the same number, and executives are subject to the effects of a group meeting (Hyndman and Athanasopoulos, 2021). If you use them, record the basis for each figure.
For scenarios, see One number or several scenarios?.
With data: read the four layers, then beat a simple rule
Six years of Shop K's sales form a time series. The first step the book always recommends is to plot it (Hyndman and Athanasopoulos, 2021). On the plot you look for four layers. A trend is a long-term rise or fall, not necessarily straight. Seasonality has a fixed and known period, such as the month of the year. A cycle is a rise and fall without a fixed frequency, usually lasting at least two years. The remainder is whatever is left (Hyndman and Athanasopoulos, 2021).
Components of a time series
An operations manager sees this quarter's sales jump and must decide on stock: how much is trend, how much is season, how much is the economic cycle?
Two points cause confusion. Cycles are often mistaken for seasonality, though seasonality follows the calendar and cycles do not. And when decomposing, the book combines trend and cycle into one "trend-cycle" component, so the four layers are a way of reading, not four separately estimated components (Hyndman and Athanasopoulos, 2021).
Before building anything elaborate, try the book's four simple rules as benchmarks: the mean, the last value, the value from the same season last year, or a line extended from the first to the last observation. The book uses Australian quarterly beer production up to the end of 2007 to forecast 2008 to 2010:
| Method | RMSE | MAPE (%) |
|---|---|---|
| Mean | 38.45 | 8.28 |
| Last value | 62.69 | 14.18 |
| Extended line | 64.90 | 14.58 |
| Same season last year | 14.31 | 3.17 |
Source: the accuracy table in chapter 5 of the book (Hyndman and Athanasopoulos, 2021). For this strongly seasonal series, the rule "take the value from the same quarter in the last observed year" beat the other three. The book says more complex methods must be compared with these rules, and if they cannot beat them they are not worth considering (Hyndman and Athanasopoulos, 2021). The step-by-step spreadsheet method for a seasonal series is in Seasonal sales forecasting in Excel, scored by MAPE; this post does not repeat it.
Which method for which situation?
The table below is my inference from the book and has not been tested on Vietnamese business data.
| Shop K's situation | Starting point | Note |
|---|---|---|
| New duvet range, no sales | Analogy, then a Delphi of about 5 people | Record the reason behind each number; compare with real sales after two months |
| Regular bedding, six years of data, clear seasonality | Same-season-last-year rule as benchmark, then a seasonal model | Use a more complex model only if it beats the benchmark on held-out data |
| A big promotion is coming that the old data never saw | Statistical forecast, then one large, documented adjustment | Small adjustments reduce accuracy |
| A shock makes history useless | Scenarios, then update on a cycle | See the case of Starbucks forecasting through COVID |
On adjusting by hand: the book says to adjust only when you have important information the model lacks, and that large adjustments tend to be more accurate than small ones (Hyndman and Athanasopoulos, 2021). People like to adjust because the forecast becomes theirs, so requiring a written reason is a way to stop unnecessary changes.
Going deeper: what did I read, and what did I not?
The post rests on chapter 6 in full and the relevant sections of chapters 1, 2, 3 and 5 of Hyndman and Athanasopoulos (2021) in the free online edition; the accuracy table comes from chapter 5, the definitions of the four layers from chapters 2 and 3, and Delphi and judgement from chapter 6. The book goes on to cover many more models (exponential smoothing, ARIMA, regression) that this post does not enter.
Dalkey and Helmer's (1963) paper on a Delphi experiment is in the source library, but only its abstract was available, so the post does not rely on it and does not use it as evidence that Delphi is more accurate than other methods; chapter 6 of the book says Delphi is slow and needs a skilled facilitator. The "by situation" table and the Shop K example are the author's inference and illustration.
Quick questions
How many experts and how many rounds does a Delphi need?
Hyndman and Athanasopoulos suggest roughly 5 to 20 experts with different expertise, and usually only two or three rounds; the more rounds, the more experts drop out (Hyndman and Athanasopoulos, 2021). An inference from the book: for a small firm the low end of that range is enough to try, as long as the experts answer independently and give their reasons.
Should I adjust a statistical forecast by hand?
Only when you hold important information the model lacks, such as a large promotion or an upcoming event. Hyndman and Athanasopoulos write that people adjust far more often than they should, that small adjustments hurt accuracy, and that adjusting to fix a pattern the viewer thinks the model missed is usually futile (Hyndman and Athanasopoulos, 2021). Record the reason for each adjustment.
How does a cycle differ from seasonality?
Seasonality has a fixed, known period tied to the calendar, such as the month or the day of the week. A cycle is a rise and fall of no fixed frequency, usually driven by economic conditions and usually lasting at least two years (Hyndman and Athanasopoulos, 2021). If your series is shorter than two years, then by my inference you will find it hard to separate a cycle from a trend.
How does a forecast differ from a target or a plan?
A forecast is the most accurate estimate you can make from all available information, a goal is what you would like to happen, and a plan is your response to both (Hyndman and Athanasopoulos, 2021). When a salesperson both forecasts and is judged against the number, the forecast tends to slide into a low target that is easy to beat.
References
Listed A–Z by author surname, Harvard style. A source with a DOI links to https://doi.org/…; one without links to its page or says plainly that it has no DOI.
- Hyndman, R.J. and Athanasopoulos, G. (2021) Forecasting: principles and practice (3rd edn). Melbourne: OTexts.No DOIotexts.comCited in 2 articles
Related articles
Data & analytics11 min readWhat a macro analyst needs: a competency map from public sources
For self-learners who want to work as, or simply read, macro analysis. The post draws on the UK government economists' standard, two forecasting reviews (Bank of England and IMF), a forecasting tournament and an employer survey, and separates what the evidence shows from my own synthesis.
- Data & analytics14 min read
Forecasting from Vietnam's CPI series: step by step on real data
For finance directors and SME owners who need an inflation assumption for a 12-month budget. The post decomposes Vietnam's real CPI series, forecasts it seven ways and scores each on held-back months. A constant-growth line is hard to beat, and the 2026 fuel surge pulls every forecast off course.
- Data & analytics9 min read
A data-driven decision loop for small businesses
For owners of small shop chains and online sellers who hold sales records but do not yet use them to choose actions. A four-step loop plus a logic model, tried on a hypothetical question: should a five-shop tea chain keep its loyalty card?
Was this article useful?
No ratings yet.
Comments (0)
No comments yet. Be the first.