GMP Data Migration

GMP Data Migration is the controlled transfer of GMP-relevant data and associated metadata between computerised systems, with documented checks of their meaning, integrity and usability for the intended use.

Related search terms include data migration validation, GxP data migration, computerised system data migration and legacy data migration. GxP is broader than GMP; not every GxP activity concerns manufacturing.

Computerised system validation addresses the wider system and its intended use. Migration-specific evidence addresses transferred records, transformation behaviour and migration acceptance—not merely whether the destination application works. The comparison below describes practical project boundaries, not universal regulatory definitions.

Moving records is not correcting their contents. As a data integrity practice, migration must not become an undocumented opportunity to overwrite legacy errors or change historical decisions. Handle discovered errors through a separate, authorised correction process that preserves the original record and the reason for change.

Start with an applicability assessment, not a generic “GxP migration” checklist. Identify the product category, jurisdiction, record purpose and system boundary before selecting controls. The table separates guidance references from binding regulatory text; it does not make every source applicable to every migration.

Within the applicable EU medicinal-product GMP context, Annex 11, section 10 recommends that changes to computerised systems, including configurations, be made in a controlled manner under a defined procedure. A practical implementation is to place the migration under change control, with named owners and authorised decisions. This implementation practice is distinct from the guidance itself; see the broader EU Annex 11 context.

Assess supplements, food, devices and chemicals separately, even when they share a platform with medicinal products. Do not derive US duties from European guidance or extend drug-GMP references across product categories. Document the basis for each source’s inclusion or exclusion, the affected record populations and who approves the assessment. Keep software support separate: a tool’s functions do not determine which legal obligations apply.

As a planning practice, inventory record populations rather than merely listing applications. For each record class, capture source systems, business owners, volumes, date ranges, lifecycle states, formats, attachments, metadata, audit history and links to other records. A batch record, for example, may depend on specifications, equipment identifiers and laboratory results held elsewhere.

Identify intended uses after transfer: active processing, historical investigation, trend analysis, batch review, inspection retrieval or archive-only access. Assign each population a disposition: transfer, retain in a linked archive, or exclude with an approved rationale. A migration disposition defines handling within the project; it is not permission to destroy data.

Before calculating reconciliation totals, define population-selection rules, extraction boundaries and treatment of in-flight transactions. Specify the snapshot or cutoff, included statuses, related-record selection and handling of records changed during extraction. These boundaries establish what the totals actually represent.

Assess risk from record criticality, transformation complexity, weak identifiers, undocumented schemas, historical defects and dependencies on obsolete software. Identify dependencies early enough to inform legacy-system decommissioning, rather than discovering them after access has been removed.

Assign named responsibilities for data ownership, technical execution, independent review, quality decisions and operational authorisation. Adapt role separation to the organisation and risk. Use software to support execution and evidence handling; authorised people make acceptance and release decisions.

Seven-step migration lifecycle

1. Approve scope, inventory, intended uses and responsibilities.
2. Specify mappings, transformations and traceability.
3. Test extraction, transfer and recovery controls.
4. Execute controlled transfers and reconcile outcomes.
5. Review exceptions and authorise cutover or rollback.
6. Confirm usable archives and authorise legacy retirement.
7. Close evidence and hand over operational responsibilities.

As implementation practice, make each mapping a testable specification, not merely a pairing of column names. Define source and target data types, field lengths, encoding, null handling, decimal precision and rounding rules. Name the owner authorised to resolve semantic differences.

Keep original creation times separate from import times; importing now does not establish contemporaneous recording of historical work. Specify time zones and how unresolved daylight-saving ambiguities are flagged rather than guessed. Treat legacy account mapping as identity interpretation: preserve historical attribution, document uncertain identities and never substitute the migration operator for the original actor. Relationship mappings should preserve structures such as lot genealogy, not just valid foreign keys.

Micro-case: assume the source records mass as 1.250 kg, the target stores grams and the conversion factor is exactly 1,000 g/kg. Then 1.250 × 1,000 = 1,250 g. Check the unit, numerical value and representation of precision separately. If the target displays 1250, preserve the source scale or equivalent precision metadata; display formatting alone does not describe measurement precision. This example establishes no universal rounding tolerance.

Include failure tests for silent truncation, blank-to-zero conversion, ambiguous dates, collapsed statuses and deduplication that removes legitimate repeated events. Specify rejection or authorised resolution instead of silent correction. Version transformation code, configuration and mapping specifications together. When a rule changes, test the affected population against that version; an earlier successful run is not evidence for the later implementation.

A practical data-integrity-by-design approach links six entities. This is an evidence architecture, not a prescribed database schema.

Maintain record-level lineage using source_system_id, source_record_id, target_record_id, run_id and mapping_version. Represent splits and merges explicitly rather than assuming one source record always produces one target record. This lineage explains migration provenance; it does not replace the historical audit trail.

For UK GxP activities within its scope, MHRA guidance section 6.13 states: “The audit trail is a form of metadata containing information associated with actions that relate to the creation, modification or deletion of GXP records.” Include that history in the metadata assessment, alongside relevant attachments, signature context and record relationships.

Within its applicable EU GMP context, Annex 11, section 9 recommends: “For change or deletion of GMP-relevant data the reason should be documented.” Connect migration-related changes to their documented reasons and authorised review decisions.

Use manifests to index files and hashes, and link execution logs, verification results and exception records through run_id. Link review decisions to the evidence reviewed; the testing section explains what hashes can and cannot show. Preserve identifiable gaps; do not manufacture historical audit events to make target history appear complete.

As implementation practice, separate four evidence questions. A successful load does not establish completeness or usability. Link checks and expected results to migration risks through a CSV traceability matrix.

Build test coverage around valid records, boundary values, malformed inputs, missing relationships, duplicate identifiers, special characters and representative legacy anomalies. Exercise each transformation path, including rejection handling; clean examples alone cannot demonstrate how the migration handles problematic data.

Track selected, extracted, transformed, loaded, quarantined and intentionally excluded populations using stable source identifiers and explicit target cross-references. Define each category and counting unit. Stage populations overlap: do not add extracted and loaded totals as though they represent different records. Within a reconciliation boundary, assign mutually exclusive final dispositions. For split or merged records, reconcile the expected mapping cardinality rather than demanding equal row counts.

Compare counts and identifier membership by meaningful strata: record class, status, site or date range. Matching overall counts can conceal a missing record replaced by a duplicate, a substituted identifier or records assigned to the wrong site. Check critical field values, parent-child relationships, attachment accessibility and metadata interpretation, including units and timestamps. Demonstrate the intended review or calculation in the target.

Use file hashes to test byte identity where files should remain unchanged. When an approved transformation intentionally changes bytes, compare content and meaning against expected results; a changed hash alone does not identify a defect.

Population-wide automated checks are practical for machine-readable identifiers, required fields and relationships. Supplement them with targeted review or sampling where interpretation is needed. Document the risk-based validation rationale using criticality, heterogeneity, transformation paths and defect detectability—not a universal sample size, fixed percentage or regulatory zero-defect formula.

Define acceptance criteria before execution. An accounted-for exception explains a record’s location; it does not resolve a defect. In the CSV deviation report, record investigation, impact assessment, correction, justified rerun scope and authorised disposition. Authorised people decide whether the evidence supports acceptance.

As implementation practice, map relevant ERP, MES, LIMS, QMS and historian boundaries, including upstream producers and downstream consumers. For each record type, name the authoritative system and accountable owner during extraction, loading and resumed operations. Include paths such as MES–ERP integration and MES–historian integration where migration could trigger operational messages.

For interfaces performing automated data capture, PIC/S PI 041-1, section 9.7, item 1 recommends validating the interface between originating, acquisition and recording systems to ensure data accuracy. This is GMP/GDP guidance, not a universal migration law.

• Sequence loads: establish reference data before dependent transactions. Restrict staging-area access, protect staged content from uncontrolled edits and distinguish staged, rejected and accepted records. Define whether each acknowledgement means receipt, validation or durable destination commit.
• Test recovery options: deterministic identifiers can support duplicate detection; checkpoints can bound restart work; idempotent processing can make repeated requests produce one intended effect. Test these behaviours under interruption and concurrent processing rather than assuming every connector provides them.
• Classify timeout outcomes before replay: rejected records need correction or authorised retry; committed records should not be blindly resent; unknown outcomes need destination lookup or investigation. Preserve original payloads, errors and acknowledgements, and link each recovery execution to its predecessor.
• Assign quarantine ownership: name who investigates, who authorises disposition and when unresolved exceptions escalate to system owners or quality personnel. An empty retry queue is not proof of delivery: compare destination identities and statuses with the transfer ledger, including quarantined and unresolved records.

Treat cutover as a controlled operational transition, not merely a successful import. The following is implementation practice for manufacturing systems: the organisation defines accountable roles, decision authority and recovery boundaries.

1. Choose the transition model. Use a freeze, phased transfer or controlled coexistence according to production constraints and system dependencies. Identify the authoritative system for each record population at every stage; prevent uncontrolled writes to both systems.
2. Build the runbook. Name execution owners and authorised go/no-go decision-makers. Specify entry criteria, write restrictions, extraction boundaries, final-delta capture, interface changes, verification steps and the recorded authorisation needed to resume work. For connected operations, include routing and queue handling in the MES–ERP integration transition.
3. Rehearse the sequence. Use a representative environment and record actual timings, failures, queue behaviour and recovery results. Adjust the plan to observed behaviour and operational constraints rather than assuming a universal downtime window or rehearsal count.
4. Establish cutover blockers. Stop progression for split writes, unexpected activity during a freeze, unreconciled final deltas or mismatched configuration versions. Assign investigation and resolution owners; repeat affected checks before seeking authorisation to proceed.
5. Define rollback before execution. Specify triggers, who can order rollback, recoverable states and how to identify, preserve and reconcile work performed after cutover. Include a fallback when returning to the old system cannot safely represent current operations.

Restoring a database or redirecting an interface is insufficient once new production records, signatures or downstream transactions exist: those events do not disappear with the restore. Account for them explicitly, preserve their history and resolve duplicates or conflicting states before restarting.

Keep technical migration acceptance separate from batch release. A successful import does not itself authorise product or quality release; those decisions remain with authorised people under the applicable release process.

For UK GxP activities within its scope, MHRA guidance, section 6.17.1, states: “Archive arrangements must be designed to permit recovery and readability of the data and metadata throughout the required retention period.” Treat this as an archive design objective, not merely confirmation that files exist.

When full original functionality cannot be retained, section 6.17.1’s format-selection guidance calls for balancing long-term accessibility against reduced dynamic functionality. Conversion to a static document is not automatically sufficient: assess which interrogation, trending or reprocessing functions remain important. The distinction between dynamic and static records helps frame that assessment.

As operational practice, compare a supported read-only legacy environment with an alternative archive using demonstrated usefulness for actual review tasks:

• Retrieve a historical record with relevant metadata, attachments and historical context; follow links between physical and electronic records rather than testing isolated file opening.
• Inventory required software, licences, encryption keys, viewers, infrastructure and specialist knowledge. Assign responsibility for keeping these dependencies usable.
• Have representative reviewers perform the intended tasks in each option. Record missing context, unavailable functions and whether those limitations prevent a defensible review.

Authorise legacy-system decommissioning only after record dispositions are completed, retrieval is demonstrated and authorised people review the retirement evidence. Document what remains accessible, where, by whom and through which dependencies. Determine retention periods from applicable record-specific rules; migration does not establish a universal retention period. Keep unresolved access dependencies visible in the retirement decision.

Both cases are illustrative, use records as the counting unit, and demonstrate implementation practice—not regulatory acceptance limits. They distinguish population accounting from successful transfer and show why recovery depends on destination evidence rather than acknowledgements alone.

Case 1: All records accounted for, but not all transferred

Assume 12,000 laboratory-result records are in scope, each with a unique source identifier. Of these, 11,970 records load successfully and 30 records enter quarantine. There are no approved exclusions, and the loaded and quarantined populations do not overlap.

• Population accounting: 11,970 + 30 = 12,000 records accounted for.
• Loaded proportion: 11,970 ÷ 12,000 × 100 = 99.75% loaded.
• Quarantined proportion: 30 ÷ 12,000 × 100 = 0.25% quarantined.

Complete population accounting does not demonstrate successful transfer of all required results. Investigate the 30 quarantined records, document their disposition through the applicable deviation process, and apply the predefined acceptance criteria. These percentages describe the run; they are not acceptance thresholds. Authorised reviewers decide whether the evidence supports proceeding. A quarantine entry establishes where a record went, not that its result is available for intended use.

Case 2: A timeout leaves uncertain outcomes

Assume a transfer contains 3,600 event records with unique source identifiers and one intended destination record per source record. Acknowledgements confirm 3,580 committed records; a timeout leaves 20 records with unknown outcomes. Destination inspection finds that 12 of those records committed and eight records are missing.

• Confirmed destination population: 3,580 + 12 = 3,592 committed records.
• Selective recovery: successfully replaying only the eight confirmed missing records gives 3,592 + 8 = 3,600 records.
• Blind retry without duplicate controls: inserting all 20 records could give 3,592 + 20 = 3,612 records, including 12 duplicates.

Using the recovery controls covered above, link the original run, destination inspection, replay selection and final identifier-level reconciliation. Retain the identities and observed status of the 20 uncertain records so reviewers can reconstruct why only eight were replayed. The final check establishes that every expected identifier occurs once; a matching total alone could conceal both missing and duplicate records.

Use this practical gate review, not a compliance certificate, with named reviewers assigned under the migration plan. Identify the exact approved versions of specifications, configurations and scripts, plus the corresponding execution evidence and approvals. An undifferentiated folder of files is not a reviewable baseline.

Judge unresolved defects against the approved acceptance criteria, recording explicit impact assessments and decisions. Keep technical completion separate from authorised acceptance: software execution cannot approve its own migration. Log remaining actions, owners and dependencies, distinguishing blockers from accepted follow-up work. For failure-mode explanations, return to the testing, reconciliation, cutover and legacy-retirement sections above. Record routine-use authorisation and legacy-retirement approval as separate decisions.