Risk & Readiness Checklist — run this before you pilot

Use this checklist to identify showstoppers and remediation tasks before you commit engineering time.

Data

• Data access: Can engineering access a representative dataset for a pilot? (If not, list required steps.)
• Data quality: Are labels, timestamps, and keys consistent? Document major gaps and expected cleaning effort.
• PII & confidentiality: Does the dataset include personal or sensitive information? If yes, plan masking, minimization, or approvals.

Technical

• Integration: What systems will the pilot read from or write to? Are APIs available?
• Performance: What latency, throughput, or uptime is required for the pilot to be useful?
• Monitoring: Can you measure key model metrics (accuracy, drift, errors) during the pilot?

Operational & Human

• Human-in-the-loop: Who reviews model outputs and how are errors corrected?
• Owner & SLA: Who owns the pilot and post-pilot handoff if successful?
• Training & adoption: Are users willing and able to use pilot outputs? Plan short training and feedback loops.

Legal & Compliance

• Regulatory review: Does the project touch regulated decisions (finance, healthcare, safety)? If yes, contact compliance/legal.
• Consent & contracts: Are there contractual limits on how data can be used?

Measurement

• Primary metric: Define a clear primary metric for success (e.g., hours saved, error rate).
• Baseline: Capture current baseline performance to compare against the pilot.
• Study design: Will you run an A/B test, time-series comparison, or before/after measurement?

Addressing these items early reduces surprises and improves the chance that a pilot will produce usable evidence. Mark unresolved items and assign owners before work begins.