Synthetic Data Risk & Utility Assessment Template

Assessor name

Who is completing this assessment?

Date

YYYY-MM-DD or free text date

Project / dataset name

Short name that identifies the dataset, model, or initiative

Dataset description

What does the dataset contain? Key tables, record counts, time span, and sensitive attributes.

Intended use cases (select all that apply)

How will the data be used? Different uses change acceptable fidelity and risk thresholds.

Model training

Share with partners / vendors

Public release

Testing / validation

Exploratory analytics / research

Benchmarking

Other

Primary analytic tasks (select all that apply)

Select the analytic tasks most important for success; use these later to define fidelity tests.

Classification

Regression

Clustering

Anomaly detection

Time series / forecasting

Descriptive statistics / reporting

Causal inference / experimentation analysis

Other

Data sensitivity level

Estimate sensitivity to guide privacy requirements.

Regulatory / contractual constraints (select all that apply)

Identify laws or contracts that affect allowable transformations and sharing.

GDPR

HIPAA

CCPA or similar

Sector-specific rules (finance, telecom, etc.)

Contractual restrictions with partners

None identified

Other

Primary privacy goals (select all that apply)

These goals will determine acceptable techniques and evaluation rigor.

Prevent re-identification

Limit attribute disclosure

Prevent membership inference

Meet regulatory minimums

Reduce overall risk below internal threshold

Privacy-preserving techniques under consideration

Select all methods you're evaluating. Later fields collect evaluation plans.

Synthetic data generation

k-anonymity / generalization

l-diversity / t-closeness

Differential privacy (DP)

Masking / tokenization / pseudonymization

Sampling / subsetting

Hybrid approach (synthetic + anonymization)

Other

Estimated baseline re-identification risk (1 = very low, 5 = very high)

Estimate risk before transformations based on auxiliary data availability and uniqueness of records.

1.0 10.0

Likelihood of privacy breach without mitigation (1-5)

How likely is an attack or accidental disclosure?

1.0 10.0

Impact of privacy breach (1 = low, 5 = severe)

Consider reputational, regulatory, individual harm, and contractual consequences.

1.0 10.0

Importance: Classification / predictive accuracy (1 low – 5 high)

1.0 10.0

Importance: Regression / numeric accuracy (1 low – 5 high)

1.0 10.0

Importance: Preserving distributions and descriptive stats (1 low – 5 high)

1.0 10.0

Importance: Feature relationships / correlations (1 low – 5 high)

1.0 10.0

Importance: Rare-event / tail behavior (1 low – 5 high)

1.0 10.0

Fidelity acceptance criteria

Define numeric thresholds that would be acceptable for key tasks (examples: max AUC drop = 0.02, MAE increase <= 10%, JS divergence < 0.05 for key features). Be specific per task.

Planned fidelity tests (select all that apply)

Choose tests that map to your primary analytic tasks.

Compare model metrics on real vs synthetic (AUC, accuracy, MAE)

Distributional distance tests (KS, JS, Wasserstein)

Correlation / covariance preservation checks

Rare event / tail representation tests

Temporal / sequential consistency tests

End-to-end downstream task tests (business KPIs)

Other

Planned adversarial / privacy tests (select all that apply)

Adversarial tests help validate claimed guarantees.

Membership inference tests

Linkage / record linkage attacks using auxiliary data

Attribute inference attacks

Re-identification simulation / risk scoring

Privacy budget and DP parameter checks (if DP used)

Other

Operational considerations: do you have lineage and access controls for the original data?

Lineage, provenance and access controls are important for safe generation and accountable reuse.

Yes

Partial

Planned generator / tool name

Name the synthetic generator, library, or service you plan to use (or 'TBD').

If using differential privacy, planned privacy budget (epsilon) or range

Enter epsilon or note 'N/A' if DP is not used. Lower epsilon = stronger privacy but lower utility.

Estimated effort for pilot (hours)

Rough estimate for running a fidelity & privacy pilot (engineering + evaluation).

Key stakeholders to notify or consult

List privacy officer, legal, data owners, security, and business owners.

Preliminary recommendation (select one)

Choose the option that best fits current risk/utility trade-offs. Use tests to validate or change this recommendation.

Recommendation justification

Briefly explain why this option was chosen, referencing risks, utility priorities, and acceptance criteria.

Next actions (select all that apply)

Practical next steps to validate and operationalize the recommendation.

Run fidelity pilot comparing real vs synthetic

Run adversarial privacy tests

Consult privacy officer / legal

Select and configure generator tool / library

Define post-release monitoring and logging

Establish data governance and reuse policies

Final confidence in recommendation (1 low – 5 high)

How confident are you in this recommendation given current information?

1.0 10.0

Notes, references, and attachments (reference links or internal ticket IDs)

Link to notebooks, evaluation results, security tickets, or storage location of artifacts.

You can explore this tool now. Sign in or create an account to save your responses and return to them later.

Make this tool part of your work

Save a personal copy, bring it to your team, or tailor the questions and workflow to fit what you are hungry to improve.

Member customization and team collaboration are coming soon.

Synthetic Data Risk & Utility Assessment Template

Synthetic Data Risk & Utility Assessment

Discussion

Services

Explore

Contact