Data Integrity Risk Assessment

Data Integrity Risk Assessment is a documented, process-based evaluation of the importance of GxP data, the ways those data could become unreliable throughout their lifecycle, and the controls and evidence needed to manage that risk.

The assessment boundary is the business process and its records—not merely an application, server or security questionnaire. It includes people, paper records, electronic records and handoffs between them. A secure application can still receive an incorrectly transcribed result or produce a report that omits decision-relevant context.

A useful assessment produces a bounded process map, a critical data inventory, reasoned risk scenarios, prioritised controls, supporting evidence and an authorised residual-risk decision. Together, these explain what could go wrong, what addresses it and who accepts the remaining risk.

Data criticality assessment is therefore an input, not a synonym: importance alone does not establish vulnerability or control effectiveness.

Start the assessment with an applicability statement: product, activity, jurisdiction and system boundary. Keep the legal basis separate from guidance used to interpret expectations, industry methods used to organise the assessment, and software used to support its execution.

Use this map to explain why each reference belongs in the assessment—not merely to compile a list of familiar documents. Do not assume identical national enforceability for Annex 11 or extend these pharmaceutical guidance rows into universal manufacturing requirements. For further context, see the guides to MHRA data integrity guidance and PIC/S PI 041. The scoring method below is illustrative practice, not a regulatory scale or prescribed acceptance threshold.

Start with the regulated decision—for example, whether a batch meets its specification—and work backwards to the observations, calculations, identifiers, units, configuration and contextual metadata needed to support it. Record the process owner, intended use of the data, decision-maker and assessment boundaries: included activities, systems, locations, external parties and exclusions with reasons.

Building on the PIC/S data-flow point in the scope table, MHRA section 4.5 extends assessment beyond the computerised system to supporting people, guidance, training and quality systems. These are guidance statements; the following mapping template is a proposed practical method, not a prescribed regulatory form.

• Map the full lifecycle: creation or acquisition, transcription, transformation, review, reporting, transfer, storage, retrieval, migration and authorised disposal. Put paper worksheets, spreadsheets, instrument files and electronic databases on the same process map.
• For every handoff, record source, destination, authoritative record, format, actor, transformation, metadata dependencies, reconciliation method and exception route. Identify how the destination is matched back to the source and who handles missing or inconsistent records.
• Attach operators, administrators, work instructions, training dependencies and quality-system interactions to the relevant steps. Include outsourced activities and routes for deviations or investigations rather than ending the map at an application boundary.

Distinguish the human-readable report from the underlying records needed to interpret it. A result summary may show the final value without its source observations, calculation version or processing context. Connect machine data acquisition to the identifiers and production context that preserve the meaning of each observation.

Example flow: sample identification → weight capture → calculation → result transfer → review → disposition decision → retained record. Assume the weight is manually re-entered into a spreadsheet and an administrator can alter its calculation template. Mark both locations for assessment; neither receives an automatic score. Show where the sample identifier, weight unit and template version travel, what the reviewer sees, and how the retained record reconnects the decision to its inputs.

In this proposed assessment method, criticality reflects the consequence of a wrong decision, not how well a record is protected. Ask what an incorrect, missing, duplicated, delayed or misleading record could change: material acceptance, process adjustment, laboratory interpretation, product disposition such as batch release, or another defined quality decision.

Include metadata and configuration wherever they determine interpretation: sample identifiers, unit-conversion factors, calculation versions, specification versions, and links connecting observations to the relevant batch or work order. Assess these dependencies together; production context tagging helps explain why an accurate value associated with the wrong batch can support an incorrect decision.

• Inherent risk: assess the scenario without crediting controls.
• Current risk: credit only existing controls supported by evidence.
• Residual risk: assess what remains after additional controls have been implemented and verified. Keep planned improvements separate from demonstrated reductions.

Stronger access controls or review do not ordinarily make the supported decision less important. They address vulnerability or the chance that a failure escapes detection. Keep consequence reasoning distinct from those control effects.

Use a consistent scenario template: “If [specified failure] occurs at [named process step] because of [stated cause], [particular data-supported decision] may be affected before [defined opportunity for detection].” Identify whether that detection opportunity occurs before the decision becomes actionable, rather than merely somewhere downstream.

Document the reasoning behind ratings, dependencies between records, uncertainty and disagreements between assessors. Record assumptions needing resolution rather than hiding them in a consensus score. Do not rank importance solely by file size, data volume, application cost or electronic versus paper format.

The complete matrix below is illustrative industry practice—not a regulatory scale, a GAMP 5 formula or a calibrated probability model. Rate each defined failure scenario from 1 to 3 for criticality (C), vulnerability (V) and escape potential (E), then calculate a dimensionless priority index: C × V × E. A higher E means poorer detection, not stronger detection.

The index ranges from 1 to 27: 1 × 1 × 1 = 1 and 3 × 3 × 3 = 27. Its limited purpose is ordering work within a consistently assessed scope. Multiplying ordinal ratings establishes neither an absolute risk level nor a percentage reduction in risk. A change from 18 to 6 is not evidence of a two-thirds risk reduction. Equal totals can also conceal different consequences; keep the three component ratings visible.

For each rating, record the scenario, rationale and supporting evidence, identifying weak evidence explicitly. Do not treat an undocumented check as demonstrated detection. If using this matrix alongside failure mode and effects analysis, document the relationship between the methods rather than assuming their scores are interchangeable.

Keep projected improvements separate from the approved residual score. Credit controls only after implementation and evidence demonstrate their effect on the relevant failure mechanism or detection pathway. Organisations must justify their own acceptance criteria; this example creates neither universal acceptance bands nor a numerical threshold for product release.

Choose controls against a specific failure mode, not a feature checklist. Prevention blocks or constrains the unwanted action; detection identifies it if it occurs. The following catalogue is illustrative practice: treat software items as requirements to evaluate, not assumed capabilities. Assign an accountable person to each owner role.

A configured feature shows a setting exists. A documented procedure tells people what to do. Neither alone demonstrates effectiveness: challenge the combined control using actual process roles, representative records and predefined acceptance criteria. Retain observed outcomes, including review records showing that the intended discrepancy was detected. Connect these tests to computerised system validation, rather than treating screenshots as proof.

Dependencies matter: attribution depends on individual identities; reconciliation depends on a complete acquisition inventory; event review depends on understood timestamps and contemporaneous recording. Test those dependencies before crediting the control with risk reduction.

For GMP/GDP operations, PIC/S guidance section 5.5.5 recommends installing and verifying appropriate procedural controls where integrated controls are technically unavailable. A practical verification should show that the assigned reviewer can detect the relevant failure under routine working conditions. Interface recovery mechanics belong in the interface and recovery assessment.

Use a practical traceability model, not a mandatory database schema. A controlled document or linked register can work if readers can follow the decision and its supporting evidence. Apply requirements traceability to connect these records:

• Assessment version → process scope and versioned data flows.
• Data flow → risk scenarios affecting particular data or decisions.
• Risk scenario → controls, ownership and residual-risk rationale.
• Control version → evidence records and review outcomes.
• Assessment decision → authorised approver, approval status, outstanding actions and next review basis.

Assign stable identifiers and explicit version references so a changed process, configuration, formula or interface does not silently inherit an earlier conclusion. Each evidence link should identify what was tested, the tested configuration, and the assessment and control versions it supports. Preserve earlier relationships rather than overwriting them.

Evidence existence is not evidence adequacy. Record test conditions, expected outcome, actual result, exceptions, reviewer and limitations. An attached file alone does not demonstrate effectiveness; the review outcome should explain whether the result supports the control claim. Link unresolved testing exceptions to the relevant validation deviation record.

One shared control may address several scenarios. Reuse its identifier, but justify evidence applicability separately for each configuration; one successful test does not establish effectiveness everywhere. Record action owners and the rationale for acceptance despite remaining limitations. Set review dates through the organisation’s review approach, not as a universal statutory interval.

Assess each interface as an end-to-end business transaction, not merely a successful message exchange. The following are proposed practice tests: challenge identity mapping, units, decimal precision and rounding, time zones, reference-data versions, partial transactions, duplicate messages, out-of-order delivery and interrupted transfers. Include timestamp behaviour alongside clock drift monitoring.

Define reconciliation before testing: match source and destination records using agreed identifiers; compare expected record counts, critical values, units, status and relationships; and account for rejected, pending or missing records. Counts alone cannot establish that the right results reached the right batch. Specify who resolves differences and what evidence demonstrates agreement.

An example exception route is to detect the discrepancy, identify affected records and downstream uses, preserve source evidence and transfer logs, investigate, obtain an authorised person's approval for correction or replay, reconcile downstream effects, and document closure. Use backward genealogy tracing where affected batch relationships need investigation. Test stable transaction identifiers and duplicate-safe replay as design considerations: replaying the same transaction should not create another business effect. Do not assume these functions exist; a retry may deliver a message without repairing an incomplete business transaction.

Restored file availability is not restored meaning. Recovery tests should check record relationships, identifiers, metadata, configuration dependencies and retrieval for the intended review. Distinguish disaster-recovery copies used to resume operation from retained records designed for later interpretation and retrieval.

For GxP activities within its scope, UK MHRA guidance section 6.17.2 recommends validating and periodically testing backup and recovery processes. Set retention periods against applicable record obligations and justify backup-testing schedules against recovery risks; this assessment method supplies no universal retention period or testing interval.

Each hypothetical case applies only the illustrative matrix defined above: priority index = C × V × E. Scores are dimensionless judgements, not measured probabilities. These cases demonstrate assessment reasoning; they do not establish sector-specific legal obligations or acceptance thresholds.

Hypothetical case 1: Pharmaceutical QC transcription

Assume a balance reading of 12.50 g is entered as 125.0 g. The entered value is 125.0 ÷ 12.50 = 10 times the source value. The overstatement is (125.0 − 12.50) ÷ 12.50 × 100 = 900%. Assume C = 3, V = 3 and E = 3: the current priority index is 27.

Evaluate direct capture, restricted editing and complete independent reconciliation before use through three challenges:

• Capture: transmit known readings, including 12.50 g, and verify decimal precision, units and sample association against the balance record.
• Tampering: attempt unauthorised edits through available entry points; confirm rejection and traceability of permitted corrections.
• Known error: seed the 125.0 g discrepancy and demonstrate that independent reconciliation detects it and prevents use until resolved.

If retained test evidence supports V = 1 and E = 1, C remains 3: the residual index is 3 × 1 × 1 = 3. The lower index is not a measured percentage reduction in actual risk. Link each challenge to its control through requirements traceability.

Hypothetical case 2: Food-process temperature interface

Assume 120 temperature observations, expressed in °C, are expected for the assessed window, but 117 unique observations arrive. Three are missing. Transfer completeness is 117 ÷ 120 × 100 = 97.5%; this percentage is not a process acceptance criterion. Assume C = 3, V = 2 and E = 3, giving 3 × 2 × 3 = 18.

Replay from preserved source records and reconcile expected identifiers, timestamps, values and units, checking for duplicates. Assume the decision supported by these data is the review of storage conditions before the affected lots are dispositioned. If complete in-process reconciliation is defined to run, and is tested to run, before that review can proceed, that independent pre-use detection may support E = 1 while V remains 2, giving 3 × 2 × 1 = 6. If reconciliation can occur only after the data are used, it is detection after use: E = 2 and the index is 12; with no demonstrated reconciliation, E = 3 and the index is 18. Replay improves recovery and detection; it does not prove transfer failures are less likely. Do not infer missing temperatures, and keep record recovery separate from the authorised person's decision about affected product.

Hypothetical case 3: Device-inspection archive

Assume the intended review needs measurement files and their work-order relationships, but a restore demonstration retrieves only PDF reports. These record needs follow from the stated intended use, not an asserted device-law provision. With C = 3, V = 2 and E = 2, the current index is 3 × 2 × 2 = 12.

Leave the index at 12 until retrieval and relationship checks demonstrate control effectiveness. Interim action: the archive owner preserves the source repository and arranges supervised retrieval before the intended review proceeds. Evidence gap: successful restoration of measurement files with verified work-order links remains undemonstrated; no lower residual score is justified.

Use this proposed practice checklist before approval. The assessment owner coordinates the record, involving the process owner and selecting quality, laboratory, engineering and IT participants according to the process—not a universal organisational chart.

• Confirm agreed scope, process-owner involvement and coverage of representative operating conditions, including exceptions.
• Check that mapped data dependencies include third parties, with responsibilities and access arrangements documented.
• Confirm each rating has a documented rationale, rather than an unexplained number.
• Assign control owners and link relevant effectiveness evidence to the scenarios those controls address.
• Identify unresolved high-consequence scenarios explicitly; record their implications for intended use and any escalation.
• Name interim-control owners and record organisation-chosen action dates, including dependencies managed through change control.
• Confirm reviewers can retrieve the necessary records, including applicable batch records, supporting data and outstanding actions.

Approval readiness means a reasoned, traceable decision with visible limitations—not a low average score, a completed questionnaire or attachments alone. Keep residual-risk acceptance and any product-disposition decision with appropriately authorised people under the organisation’s governance arrangements. Software may be evaluated to support records and workflow; it supplies neither decision authority nor the rationale for acceptance.

For medicinal-product GMP/GDP operations, PIC/S section 5.6.1 recommends periodically assessing control effectiveness through self-inspection or other review processes. In UK GxP guidance, MHRA section 4.6 recommends documenting remediation priorities, including acceptance of appropriate residual risk, communicating them to management and reviewing them. These are guidance expectations, not a universal review timetable.

As a practical approach, set an organisation-justified periodic review schedule separately from event-driven reassessment. Use change control to identify changes to formulas, interfaces, roles, intended use, supplier arrangements and migrations that could invalidate the assessment. Recurring discrepancies and failed control tests are also reassessment triggers; do not wait for the scheduled review.

Bring unresolved consequential risks, overdue remediation, recurring transfer exceptions and changed control evidence into management review. Examine whether the original assumptions still hold: does the control still work for the current process, and does its evidence support the accepted residual risk? Record the authorised decision, action owner and follow-up needed. The outcome should explain continued acceptance or further action, rather than merely renew an approval date.