Data Product & Infrastructure Spec Template — Learning-Focused

How to use this template

This template helps teams create a clear contract for any data product that supports experiments, dashboards, huddles, or learning workflows. Fill each section with the minimum detail needed so downstream teams (engineers, analysts, data stewards, experiment owners) can implement, test, and operate the product reliably.

Keep the spec versioned and discoverable in the data catalog. Link to instrumentation PRs, test datasets, and monitoring dashboards. Prefer concrete examples and a short acceptance checklist rather than long prose.

1. Use case & consumers

• Data product name: (short, unique)
• Version: (semver or date)
• Primary use case(s): (e.g., A/B testing for onboarding flows; weekly learning huddle KPI; agent signal for personalization)
• Primary consumers: (roles: Growth PMs, Data Analysts, Experimentation Platform, Huddle Facilitator, Agent)
• Consumer needs / questions this answers: (list 2–4 queries such as "Did control vs variant improve 7-day retention?", "Which cohort shows increased engagement after intervention?")
• Success criteria for the product: (e.g., metric definitions are stable, freshness < 2 hours, <1% data loss, documented ownership)

2. Product owner & catalog metadata

• Product owner: name, role, email
• Data steward / engineering contact: name, team, Slack channel
• Catalog entry / dataset location: (warehouse schema.table, table URL)
• Tags / domain: (e.g., experimentation, retention, learning-signals)
• Release notes / changelog link:

3. Metric definitions (contract)

List each metric this product guarantees. For each metric include canonical name, human-friendly description, owner, calculation (SQL/pseudocode), upstream events used, aggregation window, units, acceptable error bounds, and expected cardinality.

4. Event taxonomy mapping

Map the product's metrics back to specific events and fields. Use a small table for clarity.

Suggested columns: Event name | Semantic ID | Description | Required fields | Timestamp column | User identifier | Example payload / sample row

Example row:

<table>
  <tr><th>Event</th><th>Semantic ID</th><th>Required fields</th></tr>
  <tr><td>user_signed_up</td><td>signup.v1</td><td>user_id, created_at, source, is_test_account</td></tr>
  </table>

Note event schema evolution policy (who approves new properties or renames) and backward compatibility expectations.

5. Instrumentation checklist

Concrete verification steps implementers must pass before marking the product ready:

1. Instrumented events emitted in staging and production with the same semantic ID.
2. Event schema validated (schema registry or equivalent). Example: JSON schema, Avro schema, or column definition in CDC stream.
3. Timestamps use agreed timezone and clock source; include event_time and ingestion_time.
4. Include stable user identifier(s) and session identifier if required.
5. Test accounts / synthetic flags included to allow exclusion in analysis.
6. End-to-end test: generate known test events, run canonical metric SQL, compare result to expected value.
7. Instrumentation PRs linked and approved; changes gated by tests and monitoring updates.

6. Data quality checks & acceptance tests

Define automated checks that run on new partitions or daily. Include expected thresholds and alert targets.

• Completeness: expected rows per day ± tolerance (e.g., daily events within ±10% of 7-day moving average)
• Schema drift: detect new or missing columns
• Null rate checks: critical fields (user_id, event_time) must be < 0.1%
• Uniqueness: dedup checks for event id or composite key
• Cardinality sanity: user counts must fall within expected bounds
• Freshness check: last ingested timestamp is within SLA
• Business rule asserts: e.g., retention <= 100%, conversion funnels monotonicity

For each check include an alert destination, run frequency (partition/daily/hourly), and responsible on-call/owner.

7. Latency & freshness SLAs

Specify SLAs that match the use case. Be realistic; experimentation often needs low-latency while monthly reporting can accept longer windows.

• Ingestion latency SLA: e.g., events available in raw staging within 5 minutes 99% of the time.
• Processed table freshness: e.g., derived tables refreshed within 2 hours of event arrival.
• Backfill / recovery SLA: time to repopulate historical windows if pipeline fails (e.g., 24–72 hours depending on scope).
• Availability target: e.g., 99.5% data product availability (queries succeed, freshness SLA met).

8. Access controls & tiers

Define intended access model and any sensitive fields that require elevated permissions.

• Access tier: public / internal / restricted
• Roles allowed: Analysts, Engineers, Experimentation platform, Legal, Security
• PII / sensitive fields: list fields and masking policy
• Access provisioning process: how to request access and expected SLA

9. Retention, privacy & ethical constraints

Document retention periods, anonymization rules, and compliance constraints.

• Retention policy: raw events X days, derived tables Y days, aggregated metrics indefinite
• PII handling: keep hashed user IDs, remove or tokenise email/SSN fields
• Legal constraints: GDPR/CCPA limitations, regional restrictions
• Ethical guardrails: disallowed uses (e.g., profiling protected attributes without review)

10. Test dataset guidance

Provide instructions and a checklist for test datasets used to validate correctness and edge cases.

• Create a reproducible synthetic dataset with:
• Known cohort sizes and expected metric outcomes
• Edge cases: missing timestamps, duplicate events, out-of-order events, high cardinality users
• Test flags and a known test user id set
• Provide scripts or SQL to load synthetic data into a staging namespace
• Include expected results for canonical SQL queries (golden values) and numeric tolerances
• Document how to snapshot production sample for comparison while obeying privacy rules

11. Rollout, monitoring & incident response

• Rollout steps: staging validation → canary in production → full rollout
• Monitoring dashboards & alerts: link to DQ dashboard, freshness monitors, and metric drift reports
• Runbook excerpt: immediate steps when a DQ alert fires (who to notify, how to disable downstream consumers, how to trigger backfill)
• Post-incident review: owner, timeline, root cause, actions, and follow-up verification

12. Acceptance checklist (minimum)

1. All required events are emitted in staging & production.
2. Canonical SQL for primary metrics exists and is reviewed/approved.
3. Automated DQ checks defined and running for new partitions.
4. Freshness & latency SLAs documented and monitored.
5. Test dataset with golden results executed and passed.
6. Catalog metadata, owner, and runbook links are populated.
7. Access controls configured and PIIs handled per policy.

Appendix: Example snippet — canonical metric SQL (illustrative)

-- cohort-based 7-day retention (illustrative pseudocode)
SELECT cohort_date,
       COUNT(DISTINCT CASE WHEN activity_date BETWEEN cohort_date AND cohort_date + INTERVAL '7 days' THEN user_id END) /
       COUNT(DISTINCT user_id) AS retention_7d
FROM (
  SELECT user_id, DATE(first_seen) AS cohort_date
  FROM events.user_first_seen
  WHERE first_seen BETWEEN '2025-01-01' AND '2025-01-31'
) cohorts
LEFT JOIN events.user_activity ON events.user_activity.user_id = cohorts.user_id
GROUP BY cohort_date;

Replace with fully qualified, production-ready SQL and edge-case handling in the product docs.

Where to evolve this template

Make this template part of the organization's domain toolkit. Consider bundling related items (example: an Experiment Data Product collection with standard metric libraries, DQ pipelines, and a reusable runbook) so teams can copy and adapt with minimal friction.