Demand forecasting techniques and methods

Summary

Choose a forecasting method after defining the business decision, forecast horizon, hierarchy, data latency, and cost of error. Evaluate every candidate against a simple baseline by replaying past forecast cycles. Use the same information that would have been available on each historical cutoff date. Measure forecast quality at the level where the business acts, then monitor data, forecast, and override behavior after release. A useful forecasting system connects five parts: A specific planning decision Reliable historical and future-known data Baselines and candidate methods Backtests that reflect real operating conditions A controlled path from forecast to action Model selection is one part of the system. Data readiness, evaluation design, hierarchy, overrides, and replenishment integration often determine whether the forecast creates value.

Start with the planning decision

Write down the decision before selecting an algorithm. For each forecast, define: A daily item-store forecast may support replenishment. A monthly category forecast may support financial planning. These forecasts can use the same source transactions and still require different loss functions, horizons, and review workflows. The cost of error deserves the most care. When a stockout costs more than excess stock, the system should aim above the median forecast, and the evaluation should price that asymmetry. A symmetric error metric applied to an asymmetric decision selects the wrong method.

Target: units, orders, revenue, visits, workload, or another measurable quantity
Grain: item-store-day, category-region-week, or another business level
Horizon: the future period required for the decision
Cadence: how often the forecast runs
Lead time: how early the decision must be made
Action: purchase, allocate, schedule, price, staff, or investigate
Cost of error: the effect of forecasting above or below actual demand
Owner: the person or team accountable for the decision

Establish a data contract

Forecasting data changes over time. A live forecasting service needs to know what was available at every forecast cutoff. Create a versioned data contract that covers: Keep a snapshot or a reproducible history of each feature. A backtest becomes unreliable when it uses a promotion, price, or correction that entered the source system after the simulated forecast date. A concrete example shows why. A retailer reclassifies three stores from one region to another in March. A backtest built in June against current master data assigns those stores to the new region for the full history. The model learns regional effects that no forecaster could have known in January, and the evaluation overstates the method. Event time and ingestion time, both recorded, are the control that prevents this class of error.

Event time and ingestion time
Product, location, channel, and customer identifiers
Unit and currency definitions
Returns, cancellations, and stock transfers
Promotions and price changes
Store closures and holidays
Stock availability and lost-sales assumptions
Product launches, substitutions, and retirements
Late-arriving corrections
Future-known variables and the date they became known

Use simple baselines first

A baseline shows whether a more complex method earns its operating cost. Useful baselines include: Run every candidate against at least one seasonal baseline and the current operating method. Keep a baseline available after release as a fallback and monitoring reference. When a candidate fails live validation, the baseline carries the workload while the team investigates. A system without a fallback turns a model defect into a planning outage.

Last observed value
Same period last week or last year
Moving average
Seasonal naive forecast
Planner's current forecast
Existing operating method

Match the method to the demand pattern

Seasonal naive methods fit demand that repeats at a stable calendar interval. They are easy to explain, fast to run, and difficult to beat on some stable series. Check for a consistent seasonal period, enough history, calendar changes, promotions, and stockouts that distort the reference period. Moving averages and exponential smoothing fit relatively stable demand with level, trend, or seasonality. They work well as transparent baselines and as operating methods for many low-complexity series. Define the window length or smoothing factors, trend damping, seasonal period, and treatment of missing values and outliers. Undamped trend is a common failure: a method that extends a short-run trend without a damping factor will overshoot after the trend ends. ARIMA-family methods describe autocorrelation, differencing, and moving-average behavior. They suit time series with enough history and a useful temporal pattern. BigQuery ML provides ARIMA_PLUS for univariate forecasting and ARIMA_PLUS_XREG for forecasts that include external regressors. The documented ARIMA_PLUS pipeline can infer frequency, handle irregular intervals, interpolate missing values, adjust several outlier and holiday effects, and select an ARIMA candidate through its auto_arima option. Options such as holiday_region, data_frequency, and time_series_id_col change the result materially, so record them in the model registry rather than leaving them at defaults. These automated steps remain part of the model assumptions and should appear in the technical review. Intermittent-demand methods fit series where many periods have zero demand and nonzero events occur irregularly. Spare parts and slow-moving items often have this pattern. Croston-style methods and their variants forecast occurrence and size separately, and a single point forecast from either method understates the uncertainty. A metric dominated by high-volume items can hide poor performance on intermittent series, so report this segment on its own. Regression and gradient-boosted trees fit cases where known drivers add useful information. Examples include price, promotion, weather, event calendars, distribution, and product attributes. Build training rows from historical cutoff dates. Generate lag and rolling features using data available at the cutoff, and provide future-known inputs for the complete forecast horizon. A lag-7 feature computed with a window that crosses the cutoff leaks the future into training; the feature pipeline should accept the cutoff as an explicit input so that the same code runs during training and serving. Deep learning and foundation models belong in the evaluation after baselines and statistical methods establish the required accuracy and operating cost. They can learn across many related series and may help when the portfolio is large and the data is rich. Evaluate them under the same cutoffs, hierarchy, latency, and business loss as every other candidate. Include training cost, inference cost, retraining time, failure recovery, and skill availability in the decision. Google Cloud currently documents three forecasting paths in BigQuery: AI.FORECAST with a built-in TimesFM model, ML.FORECAST with ARIMA_PLUS, and ML.FORECAST with ARIMA_PLUS_XREG. Product status and limits can change, so verify them during implementation. Judgment and overrides add information that is absent from the data, such as a local event, range change, supplier constraint, or one-time campaign. Treat an override as structured data. Record the original forecast, new forecast, reason code, author, time, scope, expected effect, and actual result when available. Measure whether overrides improve the chosen business metric, and review repeated override reasons as candidates for new data or features.

Evaluate with rolling backtests

A single train-test split gives a narrow view. Rolling-origin backtesting simulates several historical forecast cycles. For each cutoff: Rebuild features using information available at that time. Train or update the method according to the operating schedule. Generate the full required horizon. Join the forecast to the later observed outcome. Calculate technical and business metrics. Store results by series, horizon, segment, and cutoff. Space the cutoffs to match the forecast cadence. A daily forecast earns daily or weekly cutoffs across at least one full seasonal cycle. Fewer than a dozen cutoffs cannot support a confident comparison between two close methods, because the difference between them is usually smaller than the variance between periods. Include periods with promotions, holidays, supply constraints, launches, and unusual demand. Report the distribution of error across cutoffs. An average can hide a method that fails during the most important weeks.

Select metrics that reflect the decision

Use more than one metric. Each metric has a failure mode. MAE (mean absolute error): Average error size in the target unit; Hard to compare across series with different volumes. RMSE (root mean squared error): Error with extra weight on large misses; Sensitive to a single extreme period. WAPE (weighted absolute percentage error): Total absolute error scaled by total demand; Hides low-volume series and distorts near zero demand. Bias: Systematic direction of the error; Zero bias can coexist with large absolute error. RMSSE (root mean squared scaled error): Candidate error scaled against a reference method; Needs a stable seasonal reference period. The M5 forecasting competition used a weighted scaled error across a retail hierarchy, which illustrates the importance of both scale and business weight. The strongest evaluation converts the forecast into the decision and replays the outcome. Possible measures include stockout units, waste, holding cost, fill rate, expedite cost, service-level breach, labor underuse, and overtime. Use the business loss to choose between methods with similar statistical accuracy.

Evaluate the complete hierarchy

Retail and supply-chain forecasts usually form a hierarchy. Item-store forecasts roll up to stores, categories, regions, and the total business. Define where the organization needs coherent totals. Compare bottom-up forecasts, top-down allocation, middle-out forecasts, and reconciled forecasts. An incoherent hierarchy creates a specific operational cost: the finance plan says one number, the category plan says another, and the item forecast says a third, and every reconciliation meeting spends its time on arithmetic. BigQuery ML supports hierarchical time-series identifiers for ARIMA_PLUS. Its documentation describes a bottom-up approach in which lower-level forecasts roll into higher levels. Validate the chosen hierarchy against the level where planners and systems act, and measure error at each level separately. A method can win at the total and lose at the item-store level where replenishment happens.

Segment the portfolio

Segment by volume, variability, intermittency, history length, promotion sensitivity, product life-cycle stage, margin, or service importance. Use a routing policy that assigns a method or fallback to each segment. Store the reason for the assignment. Re-evaluate routing when demand behavior changes. A seasonal review of routing catches slow drift; an annual review usually does not.

Design the path from data cutoff to planner action

A forecast used in planning needs a controlled path from data to decision. The operating flow should include: Source freshness and schema checks Feature generation at a recorded cutoff Baseline and candidate execution Forecast validation and hierarchy checks Publication to a versioned forecast table Planner review and structured overrides Delivery to replenishment or planning systems Actuals collection and delayed evaluation Drift, bias, and service monitoring Store each forecast with a run identifier, method version, training window, cutoff time, horizon, input-data version, and confidence interval where the method supports one. This record makes a later investigation possible: when a planner challenges a forecast from three weeks ago, the team can reconstruct exactly what the system knew and decided.

Failure modes after release

Four failure modes account for most forecasting incidents. The first is a silent source change. A point-of-sale system starts sending returns as negative sales instead of a separate event type. Every downstream method degrades at once, and freshness checks pass because data still arrives. Distribution checks on key fields catch this class of failure. The second is training-serving skew. The live feature pipeline computes a rolling average over a different window than the training code. The model scores normally and forecasts badly. Running the same feature code in both paths, keyed by cutoff, removes the most common cause. The third is promotion blindness. A method trained mostly on non-promoted history faces a promotion calendar and misses every event by a similar margin. The backtest must include promotions or the team will discover this failure during the first campaign. The fourth is override decay. Planners override a method that misses for a known reason, the reason ends, and the overrides continue. Measuring override value over time makes the decay visible.

Monitor three kinds of change

Watch data change first: freshness, missing values, new identifiers, history gaps, unit changes, and distribution shifts on the fields the model uses most. Watch forecast change second: bias, error by horizon, interval coverage, fallback use, and performance by segment. Interval coverage matters because a 95 percent interval that contains the actual outcome 70 percent of the time gives planners false confidence. Watch decision change third: override rate, order adjustments, service level, waste, and the time between forecast publication and action. Set review thresholds and name the owner for each alert. Keep the last safe baseline available when a candidate fails validation.

Cost, latency, and scale trade-offs

Forecasting cost follows the number of series, the cadence, and the method family. A portfolio of one million item-store series forecast daily with per-series statistical methods is a different workload from the same portfolio forecast weekly with one global model. Per-series methods scale linearly in compute and in operational attention. Global gradient-boosted and deep learning models concentrate cost in training and feature engineering, then score many series in one pass. BigQuery ML pricing ties forecasting cost to the data processed during training and inference, so training window length is a cost control as well as a modeling choice. The backtest should test whether extra history earns its price in accuracy.

Lessons from repeated forecast cycles

Three lessons repeat across forecasting programs. The baseline question changes the conversation. Teams that report candidate accuracy against a seasonal naive baseline reach a decision quickly. Teams that report candidate accuracy alone argue about whether the number is good. Leakage is a pipeline property, and reviews that only inspect model code miss it. The audit should trace one training row from source to feature and confirm that every input predates the cutoff. Forecast value shows up in the decision system, and a forecast that no system consumes has none. Trace the path from the published forecast table to the order, allocation, or schedule it is supposed to change before declaring the program complete.

Method selection checklist

What decision uses the forecast?
What grain, horizon, cadence, and lead time does it require?
Which variables are available at every forecast cutoff?
Which baseline must the candidate beat?
How does the backtest prevent future information from entering the past?
Which metrics represent the cost of overforecasting and underforecasting?
At which hierarchy levels must forecasts agree?
How will new, sparse, and intermittent items be handled?
How will planners record and evaluate overrides?
What happens when data, training, or publication fails?
Who owns monitoring and retraining?
Which customer or operational evidence supports publication claims?