Data Infrastructure for Learning — Event Taxonomy & Instrumentation Starter

Quick orientation

This starter gives a compact, practical event taxonomy and an instrumentation checklist your team can use to produce reliable, testable data for experiments and learning. It focuses on consistent event naming, required attributes for traceability and experimentation, privacy guidance, and simple QA practices so teams can trust signals used in huddles, dashboards, and A/B tests.

Why an event taxonomy matters

Raw logs and ad-hoc dashboards often disagree because events are named inconsistently, attributes are missing, ETL is flaky, or ownership is unclear. A clear taxonomy + instrumentation checklist reduces ambiguity, speeds experiments, and protects teams from noisy or misleading conclusions.

Core event types (starter set)

• user_action — A user-initiated action that matters for product behavior (e.g., click, submit, purchase).
• page_view — A user reached or rendered a page/screen (important for funnel steps).
• experiment_exposure — A user is exposed to an experiment/feature (includes experiment_id and variant).
• system_event — Non-user events such as background job completion, cache invalidation, or cron errors that affect data quality.
• conversion — Outcome events used as primary experiment metrics (e.g., purchase_complete, signup_complete).
• error — Client or server errors that may affect user experience or measurement.

Naming conventions (recommended)

Use a small, consistent set of patterns. Keep names lowercase, underscore-separated, and scoped where helpful.

• General pattern: <noun>_<verb> for actions (e.g., cart_add, checkout_submit).
• Experiment exposures: experiment_exposure and include experiment_id and variant attributes rather than encoding experiment in the name.
• Page/screen events: page_view with page_name attribute.
• System events: prefix with system_ (e.g., system_sync_complete).

Required attributes (minimal set for trustworthy experiments)

Every event should include these minimal attributes to support attribution, deduplication, and experiment analysis.

• event_id — unique id for the event (UUID) to detect duplicates.
• event_type — one of the taxonomy values (e.g., user_action).
• timestamp — ISO 8601 UTC timestamp of occurrence.
• user_id — canonical user identifier when available; use anonymous_id when not authenticated.
• session_id — session identifier to support session-level metrics.
• source — origin of the event (web, ios, android, backend-job).
• context — lightweight object for environment info (app version, locale, device).
• experiment_id and variant — when the event is related to an experiment exposure or measurement.
• value — numeric value when applicable (price, amount, duration).

Attribute conventions and examples

Keep attribute names stable across events. Example JSON payload (simplified):

{
  "event_id":"c0a80123-...",
  "event_type":"user_action",
  "timestamp":"2025-08-01T12:34:56Z",
  "user_id":"u_12345",
  "session_id":"s_abcd",
  "source":"web",
  "context": {"app_version":"1.2.3","locale":"en-US"},
  "action":"cart_add",
  "value": 29.99,
  "experiment_id":"exp_checkout_redesign",
  "variant":"B"
}

Privacy and data minimization notes

• Avoid sending PII (full name, raw email, SSN, health identifiers) in event attributes. If needed for operational reasons, store PII in a protected identity service and reference via a stable non-PII id.
• Hash or tokenise identifiers where required by policy. Document transformation steps so downstream analysts understand limitations.
• Record consent state for events when required (consent_status).

Instrumentation QA checklist (practical, run before relying on a signal)

1. Definition review — Verify the event_type, attribute names, types and semantics are agreed and documented in a lightweight spec (data contract).
2. Schema validation — Enforce a schema at ingestion (required fields, types). Generate schema errors or rejects for invalid payloads.
3. Test data — Create and run representative test cases (happy path, anonymous user, experiment variant, error path). Confirm events appear end-to-end in warehouse and dashboards.
4. Sampling and completeness — Confirm sampling rate (if applied) is documented and stable. Monitor event volumes for sudden drops/spikes versus expected baselines.
5. Lineage and transformations — Document ETL/transform steps applied to events (renames, aggregations). Maintain a simple lineage note accessible with the data product.
6. De-duplication — Verify duplicate detection logic using event_id or a composite key (user+timestamp+type).
7. Experiment integrity — For experiment_exposure events, confirm variant assignment is generated once and persisted to user profile or exposure logs to avoid double-assigning.
8. Ownership — Assign a clear owner (team/person) for the event and data product and a support contact for issues.
9. Monitoring & alerts — Create simple alerts for schema errors, volume drops, large changes in conversion rates, and failed ETL jobs.

Minimal datasets for rapid experiments

For a rapid experiment you typically need:

• Committed exposures table: experiment_exposures with user_id, experiment_id, variant, timestamp
• Event facts table: events with the required attributes listed above
• User identity table mapping anonymous_id to user_id (when available)
• A small metadata table recording experiment status & question of interest

Common pitfalls to avoid

• Using different attribute names for the same concept across events (e.g., user vs user_id).
• Encoding experiment identity in event names rather than attributes, which makes analysis brittle.
• Relying on dashboards built from transformed/aggregated tables without a path to raw events when debugging.
• No assigned owner — telemetry without stewardship decays quickly.

Implementation checklist (technical handoff)

1. Publish the lightweight event spec (taxonomy, required attributes, example payloads).
2. Implement client-side and server-side instrumentation according to spec.
3. Add schema checks at ingestion and surface schema errors to the owning team.
4. Create automated integration tests that emit test events and verify they appear in the warehouse and dashboards.
5. Document ownership, retention policy, and any PII handling rules.
6. Schedule a post-launch review to inspect volumes and metrics for the first 48–72 hours.

Next steps and templates

Copy this starter into your domain toolkit. Begin with a small set of events for the experiment or dashboard you care about. Iterate: instrument, test, run a pilot, and expand the taxonomy only when necessary.

Starter principle: prefer a small, well-documented set of dependable events over a large, inconsistent surface that nobody trusts.

If you want this as an interactive checklist: the item is a good candidate to become a saved QA checklist (run per experiment or feature) and a simple form that records test results, owner, and status. That would let teams submit QA runs and build a living history of instrumentation health.