Experiments Repository Entry Template & Evidence Bank

Purpose

This template captures the essential metadata, methods, results, decisions, and reusable artifacts for an experiment so it can be discovered, evaluated, and reused by others. Use the template as a canonical record in your Experiments Repository & Evidence Bank and adapt the required fields to your local needs.

How to use this template

Fill every required field when launching or closing an experiment. Link raw data and analysis notebooks rather than pasting large tables. Assign clear ownership and connect the experiment to the decision(s) it supports. Grade evidence quality on close and follow the archival rules described below.

Canonical fields (recommended)

• Experiment ID — stable short identifier (e.g., EXP-2026-015).
• Title — concise descriptive title that helps search (what, where, why).
• Owner / Team — person(s) responsible for design, execution, and follow-up. Include contact info or team alias.
• Date Created and Start / End Dates — planned and actual.
• Decision Link(s) — link(s) to decision records, JIRA/issue, meeting minutes, or strategy documents that motivated the experiment. Each experiment should be decision-linked when possible.
• Context & Problem Statement — short explanation of the situation, constraints, and why the experiment matters. Avoid vague framing.
• Hypothesis — a clear, testable hypothesis (if X, then Y by Z).
• Primary Metric(s) — exact metric names, units, and success criteria. Include secondary metrics and any safety or compliance guardrails.
• Variants / Treatments — describe control and all experimental variants. Include implementation notes sufficient to reproduce the setup.
• Population / Scope — who/what was included, sampling method, and exclusions.
• Sample Size / Power Considerations — planned and achieved. Note whether the experiment was underpowered and why.
• Method / Procedure — step-by-step description of what was done, dataset versions, code notebooks, scripts, and deployment details.
• Raw Data Links — links to datasets, data snapshots, analytics exports. Prefer immutable snapshots or versioned storage and include access instructions.
• Analysis Notebook / Reports — links to reproducible analysis (e.g., Jupyter/R notebooks), visualizations, and statistical tests with code and seed data where permitted.
• Results Summary — concise statement of results vs. hypothesis and primary metric outcomes with confidence intervals, p-values, or relevant statistical measures. Include a short human-readable interpretation.
• Decision Taken — link to the decision record and brief description (adopt, reject, iterate, scale, postpone). Include effective date and owners for next steps.
• Learnings & Recommendations — what worked, what didn't, unexpected outcomes, and recommended actions for reuse or further testing.
• Related Experiments — links to prior or parallel experiments to show lineage and prevent duplication.
• Evidence Quality Grade — grade (High / Medium / Low) with brief justification (see rubric below).
• Tags / Taxonomy — standardized tags for domain, objective, product, site, regulatory area, and experiment type to aid discovery.
• Retention & Privacy — retention period, redaction requirements, and data classification (confidential/public). Note any PII or regulated data used.
• Attachments & Artifacts — reusable assets such as dashboards, code templates, user flows, scripts, or SOPs that others may adopt.

Evidence quality rubric (use on close)

Grade each experiment’s evidence quality and briefly justify. This helps readers weigh reuse risk.

• High — pre-registered hypothesis, adequate power, clear causal design, reproducible analysis with versioned data and code, effects robust across checks.
• Medium — decent design but some limitations (smaller sample, partial confounder control, or single-site test). Reasonably reproducible with caveats.
• Low — exploratory, underpowered, ad hoc analysis, missing reproducibility artifacts, or known confounders. Useful for ideation but not for immediate scaling without confirmatory work.

Tagging, Linking, and Discovery

• Use a small controlled vocabulary for tags (max 6 per experiment). Example categories: objective (cost/revenue/quality), domain (manufacturing/product/service), method (A/B, pilot, simulation), and risk/regulatory flag.
• Always link experiments to decision records so evidence is actionable and traceable.
• When you copy or adapt an experiment, add a lineage tag (e.g., derived-from:EXP-2025-092).

Archival & Retention rules

• Active experiments: retain live artifacts and edit history while experiment is in flight.
• Closed experiments: preserve a read-only snapshot of raw data, analysis notebooks, artifacts, and the canonical record for at least the configured retention period (recommended 3–7 years depending on domain).
• Delete or redact regulated PII according to privacy policy; when redaction occurs, add a redaction note to the record explaining what was removed and why.

Governance & Roles

• Owner — accountable for record completeness and follow-through.
• Reviewer — optionally assigned for medium/high impact experiments to validate design and evidence grade.
• Keeper — identifies the repository steward who enforces tags, taxonomy, retention, and access policies.

Example (abridged)

Adaptation notes

Use this template as a baseline. Local teams may add required institutional fields (cost center, regulatory approvals, test environment IDs), but preserve the core fields above so experiments remain comparable across the organization.

Next steps / Checklist before publishing an experiment

1. Attach immutable data snapshot and analysis notebook or link to versioned repo.
2. Connect to decision record(s) and assign follow-up owners.
3. Apply tags and evidence grade.
4. Confirm retention & privacy settings.