Hypothesis Validation Checklist

Use this compact checklist to decide whether an exploratory finding is ready to inform decisions or needs further work.

1. Reproducibility: Can a colleague run your saved code or query and recreate the view using the same dataset and instructions?
2. Data freshness & lineage: Is the time window and source documented? Were any upstream schema or collection changes present during the window?
3. Sample independence: Is your validation using data not previously inspected (time-split, holdout, or independent dataset)?
4. Sensitivity checks: Does the signal persist when removing outliers, changing bins, or using alternative aggregations?
5. Confounder review: Have you checked likely confounders (cohorts, geography, device, seasonality)?
6. Effect size & practical significance: Are observed differences large enough to matter operationally, not just statistically?
7. Alternative explanations: Have you listed and tested at least one plausible non-causal explanation?
8. Pre-defined acceptance criteria: Did you decide in advance what evidence would count as meaningful for action?
9. Peer review: Has at least one colleague reviewed assumptions, code, and interpretation?
10. Deployment plan or pilot: If action is required, is there a small, monitored pilot or rollback plan?
11. Documentation & storage: Are findings, code, datasets, and decisions saved in a shared place with owner and priority?

Tip: run this checklist quickly after the initial exploration. Failing any box doesn't mean the observation is useless — it simply indicates what follow-up is required.