Soft Sensor Prediction

A soft sensor (virtual sensor) is a model that estimates an unmeasured or slowly measured variable—often a critical quality attribute (CQA) or state—using easily available inputs (e.g., temperature, pressures, spectra, mass flows, controller setpoints, batch context). In regulated plants, soft sensors augment PAT and MES by turning multivariate process data into real-time insight without adding invasive hardware or cycle time. Techniques range from physics-informed observers (e.g., Kalman filters) to chemometrics (PLS, PCA) and supervised learning, provided they are developed, validated, and governed under a documented lifecycle consistent with ICH Q2(R2)/Q14 and FDA PAT guidance.

Typical algorithms and signals

• Chemometrics: PLS/PLS-DA for concentration or moisture from NIR/Raman; PCA/Hotelling’s T2/Q-residuals for anomaly detection.
• State observers: Kalman/Extended Kalman/Moving Horizon Estimation for unmeasured states (e.g., bioreactor specific growth rate).
• Hybrid/first-principles: mass/energy balances with soft constraints to estimate viscosity, product temperature, or ice front.
• Supervised ML (with controls): regularized regression, ensembles; documented feature controls and explainability for GxP use.

Outputs may be used as advisory indicators, permissives/interlocks, or release-enabling evidence when supported by a validated analytical procedure and a defined control strategy. Under 21 CFR 211.68 and Part 11, computerized generation and use of predictive values must be validated, attributable, and audit-trailed; ISA‑95 interfaces define boundaries of where models execute and how data flow to MES Level 3.

Soft sensors can execute at the edge (PLC/DCS gateways, embedded analyzers), in plant historians/analytics servers, or in the MES. ISA‑95 clarifies integration boundaries: Levels 0–2 (sensing/control), Level 3 (MES operations management), Level 4 (ERP/business). The model’s runtime location dictates latency, cybersecurity posture, change control ownership, and the trust model for using predictions in execution steps or interlocks.

MES integration patterns include: subscribing to historian tags/OPC UA for features; embedding model inference as a step in the eBMR/eDHR; and writing predictions with complete context (lot/batch/phase) back to the historian/MES for CPV and review by exception. Edge buffering and store-and-forward protect continuity during network outages.

• Solid oral dosage: in-line NIR soft sensors for granule or tablet moisture and blend uniformity; use as CPP monitors with multivariate SPC limits, enabling shorter drying or blending endpoints.
• Bioprocessing: soft sensors for viable cell density, specific productivity, or titer from off-gas, capacitance, and feed rates; advisory to MPC for optimized feeding and harvest timing.
• Lyophilization: product temperature and ice front estimation from shelf temperature, chamber pressure, and MTM/smart probe surrogates to prevent collapse and reduce cycle time.
• Medical devices/combination products: viscosity/degree-of-cure estimation for adhesives or coatings (from temperature, torque, UV dose), supporting validated process windows (820/QMSR alignment with ISO 13485).
• Food/cosmetics: moisture or rheology surrogates from NIR/torque/temperature to ensure texture and microbial lethality profiles without overprocessing.

When connected to a defined control strategy, soft sensors reduce testing and cycle time while tightening process capability. FDA’s PAT framework explicitly contemplates chemometric and multivariate models to support real-time control and potentially real-time release when the analytical procedure and model lifecycle evidence are robust.

A compliant soft-sensor program documents the end-to-end lifecycle: problem definition and risk assessment; data integrity-qualified data assembly; data partitioning; model building; internal validation; independent external validation; deployment; and ongoing performance verification. ICH Q14 provides expectations for analytical procedure development, including multivariate models, while ICH Q2(R2) details validation characteristics (accuracy, precision, specificity, linearity, range, robustness) adapted for chemometric/predictive methods.

Evidence package (typical contents)

1. Intended use and regulatory impact classification (advisory, permissive control, or release-supporting).
2. Data lineage and integrity controls (source systems, time range, exclusion rules, missing-data treatment).
3. Model development report (feature engineering, algorithm choice and justification, hyperparameters, cross-validation, overfit controls).
4. Validation report (independent test set, RMSEP/MAE, bias, prediction intervals, equivalence to reference where applicable).
5. Lifecycle plan (monitoring metrics, periodic review, triggers for recalibration, change control entry points).
6. Computerized system validation (CSV/CSA) and Part 11 controls for the runtime and data flows (roles, audit trails, e-signatures).

GAMP 5 (2nd ed.) supports a risk-based approach to the computerized platform hosting models and managing inference results. Where a soft sensor is part of an analytical procedure, FDA PAT and ICH Q14 expect sound statistical design and documented robustness studies; when used for release, the model must be demonstrated suitable for its intended purpose with appropriate system suitability/monitoring.

Soft-sensor predictions are GxP records when used for in-process control or release decisions. Under 21 CFR Part 11 and 21 CFR 211.68, enforce technical controls to ensure ALCOA+ principles: secure, time-synchronized timestamping; role-based access; audit trails for model versions, parameters, and inference runs; electronic signatures for releases or deviations; and tamper-evident storage. Predictions must be linkable to batch/lot, equipment, phase, and recipe version to support complete, contemporaneous review.

• Provenance: capture model identifier, hash, and training set version at each inference.
• Traceability: write predictions back to the eBMR/eDHR with references to raw tags and calculation context.
• Review-by-exception: flag out-of-model-scope or out-of-prediction-interval events for QA assessment.
• Backup/restore and disaster recovery testing for model repositories and prediction stores.

Align MES and historian clocks and document time-base reconciliation. When third-party analytics engines are used, define the trust boundary and re-verify integrity on ingress to MES. Electronic copies and long-term retention must preserve readability and complete metadata.

Deployment decisions (edge vs server vs MES) trade off latency, cybersecurity, and change control. Establish a monitored operating envelope: model applicability domain, prediction intervals, and alarms on data drift. CPV should include model performance KPIs (e.g., RMSEP, bias, % within prediction interval) calculated per batch/lot and trended over time. Periodic verification (e.g., against reference methods) and event-driven triggers (equipment change, raw material source shift) guide recalibration under formal change control.

• Runtime health: input validation, missing-data handling, sensor sanity checks, and fallback strategies.
• Model drift: monitor distribution shifts (PCA scores, PSI, population stability) and residual trends.
• Recalibration cadence: define maximum time/volume without requalification; lock training data snapshots.
• Documentation: automatically append model version and performance summary to the executed batch record.

When predictions inform permissive conditions or interlocks, verify end-to-end latency and deterministic behavior. For advisory-only uses, ensure HMI/MES visualization communicates uncertainty and scope limits to operators.

Integrate soft sensors explicitly into the control strategy: define their role (monitoring, feedback/feedyforward, or release support), associated CQAs/CPPs, and decision rules. Under QbD, articulate how the design space and material/equipment variability are addressed by the model. Higher-impact uses (e.g., RTRT) require stronger validation, ongoing suitability checks, and defined fallbacks to reference testing upon excursions or model failure.

• Advisory monitoring: documented fitness-for-use; no direct product disposition.
• Permissives/limits: validated performance; fail-safe design; operator acknowledgment; exception handling.
• Release-supporting: validated analytical procedure per ICH Q2(R2)/Q14; system suitability; dual evidence paths; QA oversight.

Define robust, secure data flows consistent with ISA‑95. Feature inputs typically originate from Level 1/2 controllers and analyzers and are time-aligned in a historian or data hub. The inference engine publishes predictions to MES Level 3 for use in eBMR steps, permissives, and QA review. Normalized equipment and recipe context (ISA‑88 phase/unit names) improve portability.

• OPC UA tag subscriptions for signals; store-and-forward at the edge to protect against network loss.
• Historian event frames to anchor predictions to unit procedure/operation/phase segments.
• API-based model services with signed responses including model version and prediction intervals.
• MES writes back prediction plus context to support CPV and exception-based review.

Maintain segregation of duties: process engineers own model content; IT/CSV own platform qualification; QA owns use authorization and periodic review. Cybersecurity controls (whitelisting, certificate pinning, principle of least privilege) reduce risk of tampering with prediction streams.

• Data leakage and optimistic error estimates due to improper cross-validation or batch-wise splits.
• Uncontrolled upstream sensor drift causing model bias; lack of sensor metrology linkage.
• Using a soft sensor outside its applicability domain (new materials, scales, or equipment) without requalification.
• Treating advisory predictions as implicit specs, leading to undocumented de facto release criteria.
• Missing model version capture in eBMR/eDHR, breaking traceability for deviations or recalls.

Mitigations include rigorous data partitioning, sensor calibration programs linked to model KPIs, pre-defined model scope statements, and automated capture of model identifiers and inputs/outputs in the batch record. Establish clear fallbacks and operator guidance for out-of-scope events.

V5 Ultimate embeds soft-sensor lifecycle controls into execution: model registries with locked training sets; validation reports linked to QMS change control; signed model packages deployable to edge or server; and audit-trailed inference with batch/lot context in eBMR/eDHR. Predictions can drive permissive conditions, advisory steps, or RTRT evidence when justified, with QA workflows for exception review and periodic suitability checks.

• ISA‑95-aligned connectors to historians/OPC UA; event-frame mapping to unit/phase.
• Review-by-exception dashboards trending RMSEP/bias and data drift against CPV limits.
• Two-person e-signature gates for model promotion; automatic rollback on failure.
• Integrated links to deviations/CAPA when performance KPIs breach thresholds.