Risk-Based Validation: Scaling Assurance to Risk

Risk-Based Validation is a documented approach to selecting the type and depth of assurance for a computerized system or manufacturing process according to its intended use, potential failures, and consequences for product quality, patient or consumer safety, and data integrity.

The decision being scaled is practical: which functions receive attention, which failure conditions are challenged, what evidence is collected, and how closely results are reviewed. Higher-consequence failures commonly warrant deeper challenges and closer scrutiny; lower-risk uses may justify simpler assurance with a documented rationale. This allocation is not permission to disregard an applicable obligation.

The terms below describe complementary activities, not interchangeable labels. These are practical distinctions; terminology and document structures vary between organizations.

The central boundary is the object being demonstrated: evidence that software behaves correctly is not, by itself, evidence that a manufacturing process consistently produces acceptable product. Qualification can establish equipment capability, while process validation examines the manufacturing result. Risk-based validation helps allocate effort across these activities without collapsing them into a single software test package.

Start with the product and destination market, not the preferred validation method. A practical regulatory map separates applicable law from guidance used to shape an assurance strategy. It also distinguishes those sources from site procedures and industry methods: a useful method is not, by itself, a statement of legal applicability.

For US drug manufacturing, 21 CFR 211.68 carries most of the binding scaling logic for computerized systems. Paragraph (a) covers equipment generally, including computers: qualifying equipment “shall be routinely calibrated, inspected, or checked according to a written program designed to assure proper performance,” with written records of those checks maintained. Paragraph (b) is specific to computer or related systems: “Appropriate controls shall be exercised over computer or related systems to assure that changes in master production and control records or other records are instituted only by authorized personnel. Input to and output from the computer or related system of formulas or other records or data shall be checked for accuracy. The degree and frequency of input/output verification shall be based on the complexity and reliability of the computer or related system.” It also requires a backup file of entered data, or a documented, validated program where certain data is eliminated by automation, with hard copy or alternative backup that is exact, complete and secure from alteration or loss. Three obligations in (b) are fixed regardless of risk category: authorized-only changes to master records, accuracy checking of input and output, and backup of entered data. What scales with complexity and reliability is only the degree and frequency of that input/output verification — how deep the check goes and how often it runs — not whether the check, the authorization control or the backup exists at all.

ICH Q9(R1) supplies the reasoning for how deep that verification should go. Its Section 5 states that “the degree of rigor and formality of quality risk management should reflect available knowledge and be commensurate with the level of uncertainty, importance and complexity of the issue to be addressed,” and names three factors that raise required formality: uncertainty (lack of knowledge about hazards and their risks), importance (how much a decision affects product quality) and complexity (of the process or subject area). Guideline Section 3 adds that “the evaluation of the risk to quality should be based on scientific knowledge and ultimately link to the protection of the patient” and that “the level of effort, formality and documentation of the quality risk management process should be commensurate with the level of risk.” ICH Q9(R1) also states explicitly that “resource constraints should not be used to justify the use of lower levels of formality.” Applied to 211.68(b), a system with low complexity, established reliability and well-understood failure modes can justify lighter-touch, less frequent input/output verification; a system with high complexity, limited operating history or consequential uncertainty needs deeper, more frequent verification and more formal documentation of why that depth was chosen. ICH Q9(R1) is pharmaceutical quality-risk-management guidance, not a legal requirement in itself; it informs how a site reasons about the degree of verification that 211.68(b) already obligates, and the conclusion remains a documented human decision rather than a formula.

Apply a product-and-market check before choosing the framework

For supplements, food and chemicals, use a separate applicability assessment rather than assigning drug or device obligations by analogy. A practical starting point is a short applicability note that a quality or regulatory reviewer can challenge before the team commits to an assurance framework.

• Identify the product category, destination markets and manufacturing activity being assessed.
• Record the regulatory reference selected for that context and why it is relevant.
• Label each supporting document as regulation, guidance or a locally chosen method.
• Explain why any method borrowed from another sector is useful without treating that borrowing as a legal obligation.
• Route uncertain classifications or mixed-product situations to the responsible regulatory reviewer.

Keep quality risk management connected to the applicable GMP context, but document their roles separately. This makes the basis for a validation decision clearer: which source establishes the regulatory context, which informs the reasoning, and which practical choices the site has adopted.

For US medical-device production and quality-management software, FDA’s intended-use discussion recommends examining the intended uses of individual features, functions or operations. As a practical scoping method, describe an assessable function before assigning risk to an entire platform.

Keep the immediate output separate from the manufacturing decision: a comparison result is not the operator’s acceptance decision. Connect that distinction to user requirements and functional specifications. Then map the dependencies:

• Upstream inputs: instruments, master data and spreadsheet calculations.
• Downstream consumers: records, reports and connected manufacturing functions.
• Shared services: identity, time, storage and cloud services.
• Configurations: units, tolerances, permissions and interface mappings.
• Manual interventions: transcription, overrides and reconciliation across system interfaces, including APIs.

Ask: what could become incorrect, unavailable or misleading if this function failed? Include dependencies that could change that answer. This boundary exercise identifies reliance; it does not assign spreadsheets, interfaces or cloud services a universal regulatory classification.

The following practical lifecycle assigns decisions and evidence handoffs; it is not a universally prescribed sequence or approval hierarchy. Adapt responsibilities to the site's authorized roles.

1. Define intended use. The process owner describes the operational decision or action being supported, affected product or records, and excluded uses. The system owner identifies the configuration and dependencies. Output: an agreed scope against which assurance can be judged.
2. Analyze failures. Process owners and technical specialists identify how each function could fail and what follows. Output: failure paths linked to consequences, assumptions and knowledge gaps—not merely a platform-wide risk label.
3. Evaluate controls. Technical specialists identify preventive and detective controls; process owners assess whether operators can realistically apply them. Output: a control map showing dependencies, remaining exposure and which control-effectiveness assumptions need evidence.
4. Select assurance. Technical specialists propose activities that challenge the failure paths. Process owners define acceptable performance, and designated quality reviewers approve the plan before execution. Output: assurance activities, acceptance criteria established before results are observed, and named evidence owners.
5. Execute and resolve. Testers retain results against the assessed configuration and record unexpected behavior through validation deviation management. Output: evidence, investigated exceptions, dispositions and justified retesting. Do not retrospectively rewrite an expected result to turn a failure into a pass.
6. Review and authorize use. The system owner assembles intended use, configuration assessed, supporting evidence, unresolved issues and residual-risk rationale. Process and quality reviewers evaluate the package; authorized people approve, restrict or withhold use. Output: a recorded decision with conditions and accountable owners. System-use authorization remains separate from individual batch release decisions, which also remain with authorized people.
7. Maintain assurance. System owners route relevant changes through change control; process owners and quality reviewers assess their impact. Reassessment triggers include changed intended use, configuration changes, new interfaces, recurring incidents, altered process knowledge and ineffective controls. Output: an updated risk decision and a justified scope for further assurance.

Workflow: Intended use → Failure analysis → Control evaluation → Assurance selection → Execution → Review → Authorized use → Maintenance ↺ Reassessment at the affected activity.

These optional assessment methods answer different questions; choose the method that exposes the decision, not merely one that produces a rating:

• Failure mode and effects analysis: how could a function fail, and what would that failure affect?
• Hazard-oriented review: what harm could arise, where could it originate, and where might critical control points interrupt the path?
• Decision trees: which explicit conditions lead to each assurance response, and what happens when an answer is unknown?

Describe each scenario through its initiating event, affected function, downstream consequence, existing control and uncertainty. An unsupported estimate that failure is unlikely is not evidence that a control works. Challenge results, observed misses and representative operating records provide a basis for judging control performance.

Use defined qualitative categories. For example, severe means potential harm to product quality or patient safety; limited means a local disruption without that impact. Unknown likelihood means insufficient evidence, not low likelihood. Record consequence, likelihood, detectability and uncertainty separately so a reassuring label cannot conceal a weak assumption.

Before crediting detection within the control strategy, examine independence. Reusing the same calculation, database or sensor may reproduce the initiating error rather than detect it. Identify shared dependencies and challenge the detector with the originating fault present.

ICH Q9(R1) frames method selection itself as a formality decision: the guideline's Section 5 ties the degree of rigor to uncertainty, importance and complexity, not to a preferred tool. A well-understood failure with low consequence and a simple cause chain can be dispositioned through a lightweight hazard review; a poorly understood failure that is consequential or touches a complex, multi-step process warrants a more formal method such as FMEA, explicit severity/likelihood/detectability scoring and documented review. Choosing a simple method for a complex, uncertain or important failure is itself a formality decision that should be justified in the record, not an oversight to correct later.

For a severe but uncertain failure, document why unresolved assumptions justify more demanding assurance: targeted fault challenges, independent checks and review of remaining uncertainty before accepting the decision.

Select assurance activities against a specific failure hypothesis, not a target document count. The following practical examples connect the suspected failure to an observable acceptance decision; adapt criteria to the intended use.

For US medical-device production and quality-management software, FDA’s CSA guidance, Additional Considerations for Assurance Activities, allows supplier development practices, validation work and electronic information to serve as a starting point for the manufacturer’s assurance assessment.

As a practical extension of supplier qualification, compare intended-use match, software version, configuration, environment, unresolved anomalies and evidence accessibility. Translate gaps into site-specific activities: different settings prompt configuration review; different operating conditions prompt targeted challenges; inaccessible results prompt additional evidence requests or local testing. A supplier label alone does not establish that match.

For US medical-device production and quality-management software, FDA’s testing-method discussion identifies unscripted testing as potentially better suited even for high-process-risk features, functions and operations. Under this CSA approach, exploratory testing is a method choice, not a low-risk classification.

A useful evidence bundle contains the objective, assessed configuration, procedure or exploration notes, input conditions, expected behavior, actual result, anomalies and human review disposition. Use requirements traceability to connect results to intended behavior and select regression testing after changes. Retain screenshots when they demonstrate a relevant observation; indiscriminate screenshot collection cannot replace an interpretable result and a documented reviewer decision.

A practical assurance-record model links intended use to the applicable risk assessment, each credited control, the evidence evaluating that control, and the human review decision. Give each record a stable identifier and each revision a distinct reference. Evidence then points to the assessment and requirement revisions actually evaluated, rather than silently inheriting later edits.

Record the tested configuration separately from the deployed configuration: software build, settings, interfaces and relevant dependencies. Link results to versioned requirements and configuration records. Before accepting evidence for deployment, compare those configurations and document whether differences affect its applicability. A successful test against an earlier configuration is not, by itself, evidence for every subsequent configuration.

Use these practical data integrity challenges to examine whether the evidence remains trustworthy:

• Access and attribution: can reviewers identify who created, changed and approved a record?
• Time and calculations: do timestamps preserve event order, and can calculations be reconstructed from inputs, units and formula versions?
• Changes: does the audit trail preserve the history needed to interpret amended results?
• Retrieval and restoration: can recovered electronic records still expose their relationships, context and usable content?

Retain unsuccessful results and superseded decisions as history, clearly distinguished from the current accepted state. Link each superseding record to its predecessor, the reason for replacement and the person responsible for the decision. Reference related deviations, change records and operational observations from the relevant evidence or decision record; keep those wider workflows outside the core model below.

Intended use → Risk assessment → Credited control → Assurance evidence → Review decision Versioned requirements + Tested configuration → Assurance evidence Deployed configuration comparison → Review decision Predecessor record → Superseding record

A practical interface map identifies source and destination owners, record identity, quantity units, status meanings, message sequence, acknowledgements and reconciliation responsibilities. For manufacturing execution systems, distinguish acceptance of a message from authorization to act on its contents. Include rejected messages, partial processing, duplicate delivery and interrupted recovery: a successful transfer alone does not exercise these failure paths.

Illustrative case: acknowledgement lost

Assume an ERP transfers material quantities in kilograms to a manufacturing system. The receiving record controls further processing, and authorized people resolve exceptions. Connectivity fails after that record is created but before acknowledgement reaches the ERP. The following is a proposed recovery challenge, not a reported test result.

1. Retry using the original transaction identity. Challenge duplicate detection: the retry should identify the existing record rather than create another actionable quantity.
2. Reconcile source and destination identities, material, quantity, kilograms, status and processing history. Distinguish an absent record from an existing record with unfinished downstream actions.
3. Keep affected processing held while the outcome is unresolved. Route mismatches and rejected messages through authorized exception handling and reconciliation; do not silently replace conflicting data.
4. Interrupt recovery itself, then resume reconciliation. Define the observable result: the intended quantity remains represented without duplication, statuses agree, and processing resumes only after authorized resolution.

Expected evidence includes the original payload, destination record, acknowledgement history, retry and duplicate-detection events, hold behavior, reconciliation findings and the authorization record. Capture intermediate states as well as the final outcome.

Carry evidence into the process lifecycle

For US drug manufacturing, FDA’s process-validation lifecycle describes the stages below. The handoffs are practical applications, not additional guidance requirements. Apply the software-versus-process distinction covered above when interpreting the evidence.

These hypothetical reasoning exercises illustrate assurance choices, not reported outcomes or prescribed acceptance criteria.

Weighing controls

• Assumptions and units: A drug-manufacturing workflow records dispensed mass in grams and associates material identity with a batch. A separate informational display controls neither manufacturing nor disposition decisions.
• Possible failure: The workflow accepts the wrong material or permits an unauthorized override; the informational display misrepresents a value.
• Consequence: Wrong-material acceptance could affect batch composition. Display errors have lower consequence only if their effects remain isolated.
• Credited control: Assume material-to-batch checking and restricted override permissions; credit them only after challenge evidence supports their effectiveness.
• Selected assurance: Challenge incorrect identities, batch associations and unauthorized override attempts. Use focused presentation checks for the display, but also challenge shared services, data paths and resource contention that could affect the controlling function.
• Evidence needed for review: Retain challenge results, permission configurations and dependency observations. Process and quality reviewers decide whether the claimed separation justifies lighter display assurance.

Drying performance

• Assumptions and units: A drug process records temperature in degrees Celsius, elapsed time in minutes and moisture in grams per kilogram. An established product-quality endpoint informs assessment.
• Possible failure: Sensor scaling is incorrect, measurement stops, or historian records are missing.
• Consequence: Reviewers could misinterpret drying conditions, duration or progress toward the endpoint.
• Credited control: Assume independent reference measurements, interruption detection and record reconciliation.
• Selected assurance: Challenge scaling against reference inputs, interrupt measurement, and reconcile source records with the historian across recovery. Connect these results to process observations and moisture variation—not merely successful screen behavior. Scale the process-validation sampling and monitoring plan itself to the same uncertainty: where moisture endpoint behavior is well characterized from prior process knowledge, a standard sampling frequency and a narrower set of operating-condition challenges may be adequate; where endpoint variability, raw-material variation or equipment changes introduce real uncertainty, increase the number of samples per batch, broaden the operating-condition range challenged (for example, batch size, inlet air temperature or airflow extremes), and extend monitoring duration before accepting the endpoint as controlled. A pass/fail reading from a single run does not, by itself, establish that variation is controlled commensurate with its risk to the process and product.
• Evidence needed for review: Assemble reference comparisons, timestamped gaps, recovery records and endpoint observations through the MES–historian interface. Process and quality reviewers assess whether gaps leave the process-validation conclusion supportable.

For US drug manufacturing, FDA’s process-validation guidance directs manufacturers to “control the variation in a manner commensurate with the risk it represents to the process and product.” For process performance qualification specifically, the guidance recommends that the sampling plan, including sampling points, number of samples and frequency, provide “sufficient statistical confidence of quality both within a batch and between batches,” and that “the confidence level selected can be based on risk analysis as it relates to the particular attribute under examination,” with sampling during this stage “more extensive than is typical during routine production.” The drying case above applies that same logic: sampling and monitoring depth, not just a pass/fail outcome, is the lever that scales with risk.

Use these practical review checks to challenge assurance reasoning. Apply the earlier explanations of supplier evidence, testing methods and process assurance rather than duplicating them.

For recurring weaknesses, use root cause analysis to investigate why review missed them; consider corrective and preventive action where the weakness is systemic, with an effectiveness check tied to that cause.

Use these practical prompts alongside validation plans for computerized system validation. Checklist completion does not establish legal compliance or authorize product release; it supports the human decision points described in the lifecycle section.

Ready to execute assurance

• Is intended use bounded clearly, including interfaces, dependencies and operating conditions?
• Are product and jurisdiction applicability identified, with obligations distinguished from guidance and chosen methods?
• Do significant failure paths have planned evidence coverage and recorded acceptance criteria?

Ready for authorized use

• Is the tested configuration identifiable, and are anomalies dispositioned with documented reasoning?
• Do credited controls have supporting evidence, including challenges to recovery behavior where relevant?
• Are residual-risk decision owners named and the applicable human approvals recorded?

Ready to maintain the assessed state

• Are owners named for operational procedures, monitoring and periodic review?
• Are reassessment triggers defined and connected to change control, including changes to dependencies or credited controls?

Final reviewer question: can another qualified person follow the reasoning from a consequential failure to the evidence and the resulting decision?