Checklist

AI Pilot Data Readiness Checklist (Vision, PdM, Scheduling)

An interactive, practical checklist that helps teams verify whether a dataset is credible for an AI pilot (vision inspection, predictive maintenance, or scheduling). Documents size, labeling, temporal coverage, leakage and privacy checks, holdout strategy, and go/no‑go criteria.

Interactive Tool

AI Pilot Data Readiness Checklist (Vision, PdM, Scheduling)

This interactive checklist helps teams confirm whether their dataset is credible for an AI pilot. Use it to document dataset size, labeling quality, temporal and feature coverage, leakage checks, privacy/IP constraints, and a reserved holdout strategy. Answer honestly — gaps found here should be fixed before building models. Use the notes fields to capture evidence, file paths, and next steps.

Unique name, path, or table identifying the dataset.
What measurable outcome will the pilot evaluate? e.g., detect defects at X% precision/recall; predict failures Y days ahead; improve schedule adherence by Z%.
Enter the number of labeled instances (images, events, rows). Useful for estimating statistical power and training feasibility.
If vision: examples per defect type. For PdM: number of failure events per failure mode. If uneven, note imbalance strategy.
If NO, plan for resampling, weighting, augmentation, or more collection.
Includes labeling instructions, examples, and quality checks (who labeled, when, tooling).
Enter Cohen’s kappa, percent agreement, or NA if not measured.
Describe ambiguous labels, labeling drift, unclear categories, or missing label types.
How many months or cycles does the dataset span? Ensure seasonality, production cycles, and maintenance windows are included.
For scheduling: forecast horizons covered (e.g., 4 weeks). For PdM: look-ahead days available for features and labels.
For time-series/PdM ensure sampling rates, gaps, and outages are understood and documented.
Percent of records with required features present. Low completeness usually needs feature engineering or more collection.
Checks include future data in features, target leakage via timestamps, and duplicated records across splits.
Document any leaky features found and the mitigation applied (e.g., drop columns, shift windows).
E.g., 70/15/15, or temporal holdout details. For time-series, prefer temporal splits rather than random sampling.
A reserved holdout unseen during model selection provides realistic evaluation and reduces overfitting risk.
Describe selection method, size, selection dates, and storage location/path.
Useful for rare defects, extreme scheduling scenarios, or failure modes that are underrepresented.
Includes customer data, identifiable images, vendor data, and licensing of third-party datasets.
A simple baseline (heuristic, rules, or trivial model) helps judge whether the ML model adds value.
List metrics to decide pilot success (e.g., precision/recall, lead time accuracy, MAE, schedule adherence).
Concrete thresholds that determine whether to scale the pilot (e.g., >X% precision and stable for Y weeks).
Subjective summary after completing the checklist.
1.0 10.0
You can explore this tool now. Sign in or create an account to save your responses and return to them later.
Sign In to Save Create an Account
Make this tool part of your work

Save a personal copy, bring it to your team, or tailor the questions and workflow to fit what you are hungry to improve.

Member customization and team collaboration are coming soon.
Discuss Ask the Librarian

Discussion

Comments and conversation will live here.