Workbook

Forecasting Evaluation Workbook & Backtesting Examples

An interactive workbook that guides teams through practical backtesting, computes common error metrics, produces visual diagnostics, and captures decision rules that connect forecasts to operational actions. Includes templates for rolling and holdout backtests, metric selection, decomposition checks, and a place to save results and interpretation notes.

Interactive Tool

Forecasting Evaluation Workbook & Backtesting Examples

Welcome

This workbook helps you evaluate forecasting approaches so forecasts actually improve planning, staffing, inventory, and risk-aware decisions. Use it to run simple backtests, compare error metrics, capture diagnostics, and record the decision rules that convert probabilistic forecasts into action.

What you'll get

  • Guided backtest parameter capture (rolling windows, holdout, expanding)
  • Choice of error metrics (MAE, RMSE, MAPE, quantile loss)
  • Fields for pasting a small example dataset to exercise the templates
  • Spaces to record diagnostics, interpretation, and operational decision rules
  • A saved record you can reuse, extend, or export

Quick start

  1. Name the dataset and select frequency.
  2. Choose a backtest method and window sizes.
  3. Select the metrics you want to compute.
  4. Paste a small CSV sample or describe the dataset structure.
  5. Save the backtest record and use the notes to decide model replacement or operational thresholds.

Note: This workbook captures the backtest configuration and contextual notes. The platform can store your submission; advanced server-side scoring, automated charting, or direct connections to source data are described in the capability notes below.

Short identifier for this dataset (e.g., 'Store A daily demand' or 'Revenue - Product line X').
Observation frequency for the series. This helps interpret window sizes and lead times.
Choose a backtesting approach suited to your cadence and data volume.
Number of past periods used to train each model in a rolling test. Leave blank for holdout-only approaches.
Number of final periods reserved for evaluation when using a fixed holdout. Choose at least as long as the forecast horizon to avoid leakage.
Select metrics to compute when you run or export this backtest. Each gives a different view — use more than one when practical.
Select yes if your forecasts include quantiles or prediction intervals (e.g., 10th, 50th, 90th percentiles). This alters the diagnostics we recommend.
Paste a small CSV sample to exercise the templates and capture structure. For simple point forecasts use header: date,actual,forecast. For probabilistic forecasts add columns like forecast_q0.1, forecast_q0.5, forecast_q0.9. One row per date.
Select the diagnostics you'd like to note or run when automations are available.
Describe how forecast outcomes should translate into operational actions. Be concrete: include thresholds, lead times, safety stock rules, staff scheduling triggers, or escalation steps (e.g., 'If P(demand>stock) > 0.85 for horizon 14 days, create expedite order').
Summarize diagnostic findings, model comparisons, and recommended next steps. Capture confidence, known data issues, and suggested experiments (e.g., seasonal features, promotional effects).
You can explore this tool now. Sign in or create an account to save your responses and return to them later.
Sign In to Save Create an Account
Make this tool part of your work

Save a personal copy, bring it to your team, or tailor the questions and workflow to fit what you are hungry to improve.

Member customization and team collaboration are coming soon.
Discuss Ask the Librarian

Discussion

Comments and conversation will live here.