Signal Quality Engine

Real-time DSP pipeline for SCADA telemetry with filtering, drift detection, spike detection, oscillation analysis, and composite signal quality scoring.

  • Python
  • SCADA
  • DSP
Signal Quality Engine showing raw telemetry, filtered signal, and SQI dashboard

Real-time digital signal processing engine between RTU telemetry sources and downstream control logic. It filters signals, detects anomalies, and computes a composite Signal Quality Index (SQI) scored 0-100.

The engine is deterministic. It runs no background threads and no async, and timing is predictable at ~0.1-0.2ms per signal per scan. It includes a CLI, replay harness, integration adapters for the companion simulators, and an evaluation pipeline.

The diagrams use synthetic telemetry to trace one scan through the engine and into replayable quality evidence.

Signal Quality Engine system context diagram showing RTU simulator, comms front-end, SQE, and downstream logic
System context: SQE takes synthetic RTU/comms inputs and sends quality results to downstream logic, alarms, dashboards, and operator-facing consumers.

Pipeline

Each scan processes every registered signal through a fixed sequence of DSP stages:

Per-scan DSP pipeline diagram for synthetic telemetry signal processing
Per-scan pipeline: Each configured signal follows a deterministic per-scan DSP pipeline before producing quality metrics and incident inputs.
  1. Filter bank: EWMA, high-pass, and simple moving average run in parallel
  2. Innovation model: Kalman-inspired residual tracking for drift detection
  3. Drift classification: sustained, transient, and monotonic drift with configurable thresholds
  4. Spike detection: sliding-window variance with configurable sigma thresholds
  5. Frequency analysis: correlation-based and FFT oscillation detection
  6. Staleness, step-change, and plausibility checks
  7. SQI computation: weighted composite across noise, drift, spikes, oscillation, and missing samples

Signal Quality Index

SQI compresses multiple signal-health indicators into a single 0-100 score while preserving component-level evidence for troubleshooting.

Signal Quality Index component breakdown with weighted scoring model
SQI breakdown: The Signal Quality Index combines normalized component scores for noise, drift, spikes, oscillation, and missing telemetry.

Quality incidents

The incident engine consumes SQI scores and emits lifecycle events: started, updated, and resolved. Configurable persistence and hysteresis avoid flapping. Group incidents aggregate related signals, such as all points on one RTU, into a single event stream. Every incident ID is deterministic and reproducible across replay runs.

Quality incident lifecycle showing started, updated, resolved, and closed states
Incident lifecycle: Quality incidents use persistence and hysteresis so degraded signals become stable operational events instead of noisy alert flapping.

Integration layer

Three adapter modules connect SQE to the companion simulators:

  • PointCore Adapter: ingests raw point signals from the Point Lifecycle Simulator and returns processed results with quality scores
  • Scan Adapter: wraps the engine inside a PLC-style fixed-interval scan loop for deterministic simulation
  • Comms Adapter: processes signals arriving through the SCADA Comms Front-End, including comms health aggregation
Integration adapter diagram showing synthetic RTU, PLC, and comms adapter inputs to SQE
Integration adapters: Adapters isolate external protocol/data-source concerns from the core SQE processing engine.

CLI and tools

# Analyze a signal from CSV with plots
sqe analyze --signal-file data.csv --plot

# Simulate synthetic signal processing
sqe simulate --duration 1000 --noise 1.5 --plot

# Generate incident events from a signal file
sqe incidents --signal-file data.csv --out incidents.jsonl

# Deterministic replay with config
sqe replay --in scans.jsonl --config cfg.yaml --out ./output

# Offline parameter tuning
python -m sqe.cli.sqe_cli offline-tuning \
  --replay-input replay.jsonl \
  --labels labels.yaml \
  --sweep-config tuning.yaml \
  --out-dir ./tuning-results

Performance

Replay and performance evidence dashboard showing deterministic replay artifacts and benchmark outputs
Replay/performance evidence: Replay artifacts, test results, and benchmark-style outputs demonstrate deterministic behavior and bounded scan-time processing.
  • ~0.1-0.2ms per signal per scan on consumer hardware
  • Fixed memory allocation: ~15-20 KB per signal, no dynamic growth
  • Typical scan utilization: 10-20% within a 100ms PLC window
  • 146+ unit and integration tests, deterministic across runs

All visuals use synthetic telemetry and simulator-style point names. No production utility data, real station names, real HMI screens, or operational network details are represented.

SQE scores raw telemetry before control logic uses the quality results.