Experiments Repository Schema & Evidence Bank Template

Purpose

This template defines a canonical experiment record schema, attachment rules, tagging and naming conventions, and a short QA checklist so experiment evidence is discoverable, comparable, and reusable across teams. Use this template to capture experiments consistently, speed learning, avoid duplication, and connect experiments to decisions and downstream work.

How to use this template

1. Create one record per discrete experiment (a controlled test, pilot, or intentional change with measurable outcomes).
2. Fill the canonical fields below. Attach raw data, analysis notebooks, images, recordings, protocols, and final reports in the Evidence section.
3. Apply tags and link related experiments or decisions so others can discover your work.
4. Run the QA checklist before marking the experiment as Completed or Archived.

Canonical Fields (required)

• ExperimentID — Stable unique id (prefixed by team or unit), e.g. EXP-OPS-2026-014.
• Title — Short, descriptive title that includes the outcome type (e.g., "Reduce pick error rate: barcode confirmation pilot").
• Short Summary — One-sentence outcome-oriented summary (what you tested and why).
• Hypothesis — Clear, testable hypothesis (including expected direction and magnitude where possible).
• Primary Metric(s) — Name, definition, units, baseline, target, and measurement frequency.
• Owner — Person accountable for the experiment record and communication.
• Team / Sponsor — Team(s) involved and senior sponsor if applicable.
• Start Date and End Date — Planned and actual.
• Status — Planned, Running, Completed, Stopped, Scaled, Archived.
• Scope — Where (locations, product lines, user segments) and who was affected.
• Design — Brief description of the experiment design (A/B, pilot, before/after, randomized, observational). Include key variables and controls.
• Data Sources — Where measurements come from (databases, sensors, surveys), sample size, and any data-cleaning notes.
• Artifacts / Evidence — Links to attachments: raw data, analysis notebooks, dashboards, photos, videos, SOPs, consent forms, protocols. Use stable URLs or internal object IDs.
• Results — Numeric outcomes for primary and secondary metrics, statistical significance or confidence intervals when relevant.
• Interpretation — What the results mean in plain language. Include caveats, limitations, untested assumptions, and alternative explanations.
• Decision — Action taken: Adopted, Rejected, Modified, Scaled, Needs more data. Record who decided and when.
• Next Steps — Recommended follow-up actions and owners (scale plan, SOP update, further experiments).
• Related Experiments / References — Links to predecessor/successor experiments, literature, or official standards.
• Tags / Taxonomy — Standardized tags for discovery (see tagging rules below).
• Version — Record version and date of last update.

Recommended Optional Fields

• Cost / Effort — Estimated budget and person-hours.
• Risks / Safeguards — Short description of potential harms and mitigation plans.
• Customer Impact — Who is affected and how (internal customers, external customers, regulators).
• Confidence Score — Reasoned confidence (e.g., 1–5) about result reliability.

Attachment & Evidence Rules

• Attach raw data and analysis where practical. Never attach sensitive PII without appropriate access controls.
• Name attachments with a clear prefix: ExperimentID_Type_YYYYMMDD (e.g., EXP-OPS-2026-014_rawdata_20260511.csv).
• Prefer stable, internal links (object IDs) to local file paths. Include a README when analysis requires specialized steps to reproduce.
• When using external tools (Jupyter, Looker, Tableau), capture a static snapshot (PDF) plus a link to the live dashboard.

Tagging & Naming Conventions (for discoverability)

Use a small controlled vocabulary to make search and filtering effective. Suggested tag categories:

• Domain / Function: ops, sales, quality, product, hr, safety
• Method: ab-test, pilot, case-study, observational, mvp
• Outcome Type: cost-reduction, defect-rate, lead-time, satisfaction
• Technology / Tool: iot, rpa, ml, mobile-app
• Scale Intent: experiment, scale-ready, operationalized

Example tags: ops, pilot, defect-rate, barcode-scan, scale-ready

Short QA Checklist (before marking Completed / Archived)

1. Is the hypothesis stated clearly and measurable?
2. Are primary metrics defined with baseline and units?
3. Is the raw data attached or a stable data source link provided?
4. Is the analysis reproducible (notebook, script, or documented steps)?
5. Are limitations, biases, and caveats documented?
6. Is a documented decision recorded (Adopted/Rejected/Other) with owner and date?
7. Are related experiments and downstream owners linked for handoff?
8. Are tags applied from the controlled vocabulary?

Governance & Access

Assign clear ownership for experiment records. Sensitive experiments should have restricted access and a documented retention policy. Keep an index of experiment IDs and a lightweight approval process for publishing experiments that will be widely shared.

Example Record (abridged)

ExperimentID: EXP-PICK-2026-003

Title: Barcode confirmation pilot to reduce pick errors

Hypothesis: Adding barcode confirmation before package sealing will reduce pick error rate by >30% within 30 days.

Primary Metric: Pick error rate (% incorrect items per pick) — baseline 2.8%, target 1.9%.

Design: Pilot in one shift at Plant A for 30 days. Pre/post comparison with control shift.

Artifacts: EXP-PICK-2026-003_rawdata_20260501.csv; EXP-PICK-2026-003_analysis_20260515.ipynb; PDF snapshot of dashboard.

Result: Error rate dropped to 1.6% (p<0.05). Decision: Scale to all shifts; Owner: Ops Director; Next step: update SOP and schedule roll-out.

Discovery & Surfacing

Index experiment records in a searchable repository. Build simple saved filters for common queries (by tag, owner, metric, status). Consider a monthly "experiment highlights" digest for cross-team learning.

When this template is not enough

For very large-scale experiments, regulated environments, or clinical research, extend the schema to include consent tracking, protocol approvals, audit trails, and regulatory compliance metadata.

Next improvement opportunities

• Create an Interactive submission form so teams can create experiment records from a guided interface and store structured entries for search and dashboards.
• Build a lightweight dashboard that shows running experiments, recent results, and decisions to help governance and scaling.