Learning Ops Playbook — Roles, Cadences, and Minimum Viable Processes

Welcome — Why Learning Ops Matters

Many organizations run experiments, pilots, and one-off improvements. Too often the lessons stop with the project team. Learning Ops turns experimentation into a predictable capability that consistently produces usable knowledge, measurable impact, and durable change. This playbook gives a minimum-viable structure you can adopt quickly: roles people can own, repeatable cadences to keep work moving, an experiment lifecycle with evidence standards, simple handoff rules, and governance light enough to scale.

Core Hunger Served

Help teams turn isolated experiments into scalable organizational learning so improvements spread, decisions get better, and value compounds across projects.

Quick Orientation

This playbook is intentionally lightweight. Use it to stand up a Learning Ops practice at the team or site level, then iterate. Keep the practices visible: an experiment registry, recurring cadences, and a short evidence checklist ensure lessons are captured and reused.

Role Definitions (Minimum Viable)

• Learning Lead — Owns the Learning Ops capability. Maintains the experiment registry, runs cross-team cadences, helps prioritize experiments, and curates reusable learnings.
• Experiment Owner — Responsible for running a specific experiment end-to-end: hypothesis, design, execution, data collection, and write-up. Owns the decision at experiment maturity (adopt/iterate/stop).
• Data Steward — Ensures data integrity and accessibility. Defines metrics, measurement windows, guardrails, and confirms evidence meets standards. Helps automate data capture where possible.
• Operational Owner — The day-to-day owner who will take an adopted change into production or operations. Engaged before the experiment begins to advise handoff feasibility and risks.
• Stakeholder Sponsor — Provides prioritization support and can unblock resources or approvals. Not required for every small experiment; needed for experiments with cross-team impact or budget implications.

Minimum-Risk RACI (Sample)

Map these responsibilities to your local roles. Use this as a starting point:

• Experiment Intake: R=Learning Lead, A=Stakeholder Sponsor, C=Operational Owner, I=Experiment Owner
• Design & Measurement Plan: R=Experiment Owner, A=Stakeholder Sponsor, C=Data Steward, I=Learning Lead
• Execution: R=Experiment Owner, A=Operational Owner, C=Data Steward, I=Learning Lead
• Analysis & Decision: R=Experiment Owner, A=Operational Owner, C=Data Steward, I=Learning Lead/Sponsor
• Handoff to Ops: R=Operational Owner, A=Learning Lead, C=Experiment Owner, I=Data Steward

Meeting Cadences & Templates

Cadences keep experiments moving and make learning visible. Keep meetings short and outcome-focused.

• Weekly Experiment Huddle (30–45 min) — Participants: Experiment Owners, Learning Lead, Data Steward (optional). Agenda: quick status updates (3–5 minutes per experiment), blockers, decisions needed this week.
• Monthly KPI Review (60 min) — Participants: Learning Lead, Data Steward, Operational Owners, Sponsors. Agenda: KPI trends across experiments, evidence quality, experiments ready for adoption.
• Quarterly Learning Review (90 min) — Broader stakeholders. Agenda: portfolio-level outcomes, cross-pollination opportunities, capability gaps, and backlog prioritization.
• Retrospective Cycle (after an experiment completes) — Quick retro (30–60 min) with the team to capture what worked, what didn’t, and next experiments inspired by the outcome.

Experiment Lifecycle (Practical Steps)

1. Intake — Submit an Experiment Brief into the registry. At minimum include: hypothesis, desired metric, primary metric, guardrails, estimated effort, timeline, and operational owner.
2. Prioritize — Learning Lead or a small prioritization panel triages experiments weekly using simple criteria: potential impact, confidence gap, cost/effort, and risk tier.
3. Design & Measurement Plan — Define success metrics, data sources, minimum run-time, sample size rules (practical guidance below), and qualitative feedback mechanisms.
4. Run — Execute the experiment per plan. Log events and interim observations into the experiment record. Escalate blockers to the Learning Lead.
5. Analyze — Data Steward and Experiment Owner review evidence against the measurement plan and guardrails. Summarize outcomes and limitations.
6. Decide — Outcome options: Adopt, Iterate (new hypothesis), or Stop. Document the decision, rationale, and next steps in the registry.
7. Handoff & Codify — If Adopted: update SOPs, train Operational Owner, instrument monitoring, and store a concise playbook or checklist for the change.

Minimum Evidence Standards

Keep standards pragmatic so evidence is useful without becoming a research project.

• Measurement Plan — Must state: primary metric, baseline, measurement window, data sources, and at least one guardrail metric.
• Run Duration — Default to a minimal window that captures normal variability (commonly 2–6 weeks); use shorter pilots for qualitative learnings only.
• Data Quality — Data Steward must confirm data accuracy and completeness before analysis is accepted.
• Decision Record — Every decision must include the evidence summary, who made it, and what conditions would trigger re-evaluation.
• Reproducibility — Document the steps required to re-run the experiment; include configuration, sample selection, and scripts or queries where relevant.

Practical Measurement Rules (rules of thumb)

• Prefer effect-size and direction over strict p-values for operational experiments — ask: is the observed effect clearly material and actionable?
• For small samples use multiple runs or qualitative evidence annotations rather than overstating confidence.
• Record both signal and noise: show the baseline distribution, not only the averages.

Handoff Criteria — When an Experiment Is Ready for Operations

Before marking an experiment as Adopted, confirm the following checklist:

• Operational Owner assigned and accepts responsibility.
• SOP or runbook drafted and stored in the team knowledge base.
• Monitoring/alerts in place for primary metric and guardrails.
• Rollback or mitigation plan exists and is tested conceptually.
• Data pipelines and reports are automated (or scheduled) for ongoing monitoring.
• Training or comms plan for affected teams is prepared.
• Compliance or safety sign-offs completed if applicable.

Lightweight Governance

Governance should prevent major risks without slowing small experiments. Use a tiered model:

• Low-Risk — Team-level experiments. Approval by Learning Lead; run and decide locally.
• Medium-Risk — Cross-team impact or >$X budget. Require Operational Owner and Sponsor review before execution.
• High-Risk — Safety, regulatory, or significant financial exposure. Require explicit Sponsor approval and governance checklist completion.

6-Week Onboarding Plan to Stand Up Learning Ops (Minimal Viable Launch)

1. Week 1 — Orientation & Registry
   • Introduce the playbook to key stakeholders.
   • Create an experiment registry (simple shared doc or tool) and an Experiment Brief template.
   • Assign a Learning Lead and identify initial Experiment Owners.
2. Week 2 — First Intake & Prioritization
   • Run an intake session; submit 2–4 candidate experiments into the registry.
   • Prioritize using a simple matrix: impact × effort × risk.
3. Week 3 — Measurement Plans
   • Data Steward helps each Experiment Owner create a measurement plan for one pilot experiment.
   • Schedule the weekly huddle cadence.
4. Week 4 — Run Pilot & Log Evidence
   • Execute the first pilot and log observations in the registry.
   • Use the weekly huddle to surface blockers and small wins.
5. Week 5 — Analyze & Decide
   • Analyze pilot evidence with the Data Steward.
   • Make a documented decision: adopt, iterate, or stop.
6. Week 6 — Handoff, Retro & Roadmap
   • If Adopted, start the handoff checklist and create a short playbook for operations.
   • Conduct a retrospective on the Learning Ops launch and capture improvements for the next 6-week cycle.

Essential Templates & Artifacts

Keep these concise. Examples to create and store in your knowledge base:

• Experiment Brief — Title, owner, hypothesis, primary metric, baseline, guardrails, estimated effort, timeline, operational owner.
• Measurement Plan — Data sources, measurement windows, acceptance criteria, sample-size guidance, and known data limitations.
• Weekly Huddle Agenda — 5-minute updates per experiment, 10 minutes for decisions/blockers, 5 minutes actions.
• Decision Record — Short summary: outcome, evidence, decision maker, date, re-evaluation conditions.

Useful KPIs for Learning Ops

• Experiments started per quarter
• Percentage of experiments with complete measurement plans
• Adoption rate (adopted / completed experiments)
• Average time from intake to decision
• Number of codified playbooks created

Common Failure Modes & Mitigations

• Lost Lessons — Mitigate: require a one-page Decision Record and store it in a searchable registry.
• No Ownership — Mitigate: assign Operational Owner before experiments begin; require signoff for adoption.
• Poor Measurement — Mitigate: Data Steward reviews and approves measurement plans before execution.
• Scope Creep — Mitigate: keep experiments bounded and time-boxed; use the weekly huddle to re-scope quickly.
• Over-Centralization — Mitigate: keep approval lightweight for low-risk experiments and delegate to teams.

Next Steps & Suggestions for Tailoring

Start small. Use this playbook for a 6-week pilot in one team or site, then adapt the cadence, roles, and evidence bar based on your industry, compliance needs, and risk tolerance. As your capability matures, you may add automated experiment registries, dashboards, and richer analytics.

Where to Start Right Now

1. Create a one-page Experiment Brief template and a shared experiment registry.
2. Schedule a 30-minute launch meeting with a Learning Lead, a Data Steward, and two Experiment Owners.
3. Run your first intake and commit to the weekly huddle cadence for 6 weeks.

Appendix — Lightweight Risk Triage

Tag experiments as Low / Medium / High risk during intake. Use Sponsor involvement only for Medium/High tiers. Keep the triage criteria visible in the registry.