Predictive Maintenance — Data Collection & Labeling Plan (Template & Checklist)

Purpose

This template helps pilot teams collect the right combination of sensor, operational, and labeled event data to develop, validate, and evaluate practical predictive maintenance models. It focuses on producing reliable, testable datasets that support meaningful metrics (precision, recall, time-to-detection, lead time) and avoid common data mistakes that produce unusable or misleading models.

How to use this template

Copy the checklist and the example spreadsheet schema into your pilot workspace. Use the asset list and criticality scoring to prioritize where to collect data first. Define concrete labeling rules before collecting manual or historical labels. Run the quality checks on a sample dataset before committing to a full pilot. Tie the pilot evaluation metric targets to observable operational actions (who will respond to alerts, what actions are expected, how outcomes are measured).

1. Asset list & criticality

Create a canonical asset inventory for the pilot scope and assign a simple criticality score so you focus data collection where it delivers the most value.

Sort candidate assets by ConsequenceScore × likelihood (if available) to prioritize pilot data collection.

2. Candidate sensors & suggested sampling rates

List existing and required sensors, their sampling frequency, and why each signal matters.

When uncertain, over-sample short durations initially (higher frequency) to validate signal usefulness, then down-sample or extract features for long-term storage.

3. Event labeling rules (clear, testable, repeatable)

Define label types before labeling data. Use specific rules that anyone on the team can apply to historical records or flag in real time.

Common label types

• Binary failure label: 0 = normal, 1 = failure event. Define the exact start and end criteria (e.g., 'failure start = first recorded pressure drop >20% sustained for 5 min; failure end = repaired and back to within 5% of nominal for 30 min').
• Degradation stage labels: normal / warning / critical with precise thresholds.
• Remaining Useful Life (RUL): time until next corrective action or component replacement. Use only where reliable maintenance records exist.

Labeling rules — examples

1. Define failure root cause categories (bearing, lubrication, electrical). Each event must be assigned one or more root causes based on maintenance records and technician notes.
2. Time anchoring: All labels must include LabelTimestamp and LabelAuthor. If derived from a work order, include WorkOrderID and actual start/end timestamps.
3. Minimum evidence: Require at least two corroborating sources (sensor excursion + maintenance record OR technician inspection + alarm log) to confirm a label.
4. Label scope: Assign whether label applies to an entire asset, a component, or a sub-system. Be explicit about AssetID and ComponentID.
5. Negative/near-miss labels: Consider labeling near-failures (interventions before failure) for supervised learning of early-warning states.

Document edge-cases (e.g., intermittent signals, transient trips) and how labels are resolved. Keep a short 'label decisions' log for auditability.

4. Spreadsheet / dataset column schema (example)

Use a consistent timestamp, canonical asset IDs, and clear types. Below is a suggested CSV column set for model-ready datasets.

Keep raw high-frequency data separate (raw waveforms or ringbuffers) and store derived features (RMS, kurtosis, spectra bands) in the main modeling table to reduce size and complexity.

5. Storage, timestamping, and retention guidance

• Store all timestamps in UTC and include timezone metadata for systems that report local time.
• Preserve raw sensor data for at least the pilot period (recommended: 6–12 months) so you can reprocess features. Store derived features for longer if useful.
• Keep a mapping table that links SensorID > AssetID > ComponentID with calibration metadata and units.
• Version data extracts and label sets. When labels or mappings change, retain older versions for reproducibility.
• Check regulatory, privacy, or IP requirements before exporting or sharing datasets externally.

6. Data quality checks (minimum set)

Implement automated checks that run at ingest and periodically on stored data:

1. Missing data rate: percent of timestamps without at least one required sensor value (e.g., flag >5% per day).
2. Stale sensor detection: no new samples for a sensor for > expected interval.
3. Out-of-range values: values beyond physical or expected limits.
4. Sensor drift / calibration changes: moving mean shifts over a baseline period.
5. Label consistency: check that each labeled failure has matching maintenance or workorder evidence.
6. Class imbalance report: percent of failure vs normal samples (helpful to plan sampling or augmentation).

Capture Q/C failures as metadata fields (QC_Flag, QC_FailReason) for downstream filtering and model training decisions.

7. Basic pilot evaluation metric table & definitions

Pick metrics that reflect operational usefulness, not just classic model scores.

Define the acceptance criteria and the minimum dataset size (number of failures, hours of normal operation) needed for statistically meaningful evaluation.

8. Implementation checklist (step-by-step)

1. Assemble cross-functional pilot team: maintenance, operations, data, and IT; assign owners for labeling and data governance.
2. Complete asset inventory and criticality scoring for pilot scope.
3. Inventory existing sensors and plan any temporary instrumentation for the pilot.
4. Agree and document precise labeling rules and evidence requirements; run a small labeling pilot on historical incidents to calibrate rules.
5. Define data storage, access, and retention policy; set up ingest pipelines and timestamp normalization.
6. Collect an initial dataset (recommended minimum: X failures or Y months — define per pilot) and run the data quality checks.
7. Create derived features and a baseline model (simple thresholds or logistic model) to set a performance baseline rather than starting with a complex black box.
8. Evaluate model against the pilot metric table. Include human review of false positives/negatives to refine labels and features.
9. Confirm governance and alerting actions: who gets notified, what steps follow an alert, and how to log actions.
10. Document results, lessons learned, and next steps (scale, integrate alerts into CMMS, or iterate on instrumentation and labeling).

9. Governance, ethics & anti-overhype guidance

• Start small: pilot on a few prioritized assets before attempting plant-wide rollout.
• Measure operational actionability, not just model metrics. A precise model that produces un-actionable alerts will erode trust.
• Capture human-in-the-loop feedback: allow technicians to mark alerts as useful/useless and feed that back into labeling or thresholds.
• Avoid post-hoc label leakage: ensure labels are derived only from signals available at prediction time, not from repair reports that include post-failure tests.
• Keep a clear record of model versions, data versions, and label decisions for audits and continuous improvement.

10. Example CSV snippet (first 3 rows)

AssetID,ComponentID,Timestamp_UTC,Vib_RMS_X,Temp_Sensor1,OperatingMode,FailureLabel,LabelTimestamp,Source
PLT1-CMP-A1,bearing-1,2024-02-01T00:00:00Z,0.012,45.3,run,0,,sensor
PLT1-CMP-A1,bearing-1,2024-03-12T03:41:00Z,0.178,62.1,run,1,2024-03-12T03:30:00Z,workorder#3456
PLT1-CMP-A1,bearing-1,2024-03-12T03:50:00Z,0.201,64.4,run,1,2024-03-12T03:30:00Z,tech-inspect
  

Use

Attach this template to predictive maintenance pilots to reduce common data mistakes, standardize labeling, and set clear success criteria. Adapt sampling, label rules, and retention based on asset type and organizational constraints.

Notes & adaptation tips

• If failure rates are extremely low, consider targeted stress testing or seeded fault experiments to produce labeled examples safely.
• When high-frequency waveform data is large, store waveforms in a raw archive and keep pre-computed features for model training.
• Run a small 'labeling calibration' exercise where multiple technicians label the same events to measure inter-rater agreement and refine labeling rules.
• Keep this template under version control within your domain so future pilots benefit from earlier labeling decisions and schema improvements.