The EU AI Act for GxP Manufacturing: What Pharma, Medical-Device and Biotech Quality Leads Actually Have to Do

The AI Act regulates AI systems and general-purpose AI (GPAI) models placed on the EU market, put into service in the EU or whose output is used in the EU — regardless of where the provider sits. It uses a risk-tiered structure: prohibited practices (Article 5 — social scoring, real-time biometric ID with narrow exceptions, manipulative AI), high-risk AI systems (Article 6 + Annex I and Annex III), limited-risk systems with transparency obligations (Article 50 — chatbots, deepfakes, emotion recognition), and minimal-risk systems with no additional obligations. The two definitions that drive most GxP scope are in Article 6: (a) AI systems used as a safety component of a product covered by Annex I Union harmonisation legislation — which includes the Medical Devices Regulation (EU) 2017/745, the In Vitro Diagnostic Regulation (EU) 2017/746 and the Machinery Regulation, but not the human-medicines or veterinary-medicines acquis — and (b) AI systems falling into the use-case categories in Annex III (which includes employment/HR, education, essential services, law enforcement and several others). A pharma manufacturer using AI to triage adverse-event reports, schedule preventive maintenance or optimise CPV is generally not in Annex I or Annex III scope; a medical-device manufacturer using AI inside a Class IIa device, or any manufacturer using AI in hiring/promotion of workers, is.

If an AI system is in scope of Article 6 (high-risk), the provider obligations in Chapter III Section 2 are substantial and overlap heavily with what a GxP-mature manufacturer already does — but they are not identical. The headline obligations: a risk management system across the lifecycle (Article 9), data and data governance for training/validation/test sets (Article 10), technical documentation per Annex IV (Article 11), automatic logging across the lifecycle (Article 12), transparency and instructions for use (Article 13), human oversight (Article 14), accuracy, robustness and cybersecurity (Article 15), a quality management system (Article 17), automatic event logging (Article 19), corrective actions and duty of information (Article 20), cooperation with competent authorities (Article 21), registration in the EU database (Article 49), conformity assessment (Article 43) and an EU declaration of conformity plus CE marking (Articles 47/48). Deployer obligations (Article 26) cover instructions-for-use compliance, human oversight, input-data appropriateness, monitoring and incident logging. A manufacturer that is both provider and deployer (most are) carries both stacks. The risk-management, QMS and technical-documentation obligations map cleanly onto ICH Q9(R1) and ISO 13485 / IEC 62304 / Annex 11 / GAMP 5 — they do not replace them.

Article 10 of the AI Act is the data-governance article, and it is where most pharma and medical-device manufacturers will spend the bulk of their AI-Act readiness effort. The requirement: training, validation and test data sets must be subject to data-governance practices appropriate for the intended purpose, including relevance, representativeness, freedom from errors, completeness, statistical properties, examination for biases, identification of data gaps and the measures taken to address them. This aligns closely with the FDA / Health Canada / MHRA Good Machine Learning Practice (GMLP) ten guiding principles (October 2021) and the FDA's January 2025 draft guidance on AI/ML-enabled device software functions, which together push for data-pipeline documentation, dataset-level traceability, bias evaluation, performance monitoring and a Predetermined Change Control Plan (PCCP) for model updates. For Annex 11 / 21 CFR Part 11 systems, the same data has to satisfy ALCOA+ data-integrity expectations — the AI Act adds bias and representativeness on top of attributability, legibility, contemporaneousness, originality and accuracy.

One of the practical problems the AI Act inherits from medical-device practice is what to do about models that learn or are updated after market placement. The AI Act addresses this in Article 43(4): a substantial modification to a high-risk AI system can be excluded from a new conformity assessment if the change is pre-determined in the original technical documentation. This mirrors the FDA's PCCP framework (final guidance December 2024) for AI/ML-enabled device software functions and the IMDRF's predetermined change-control work. The pattern: at filing, define the change envelope — what can be updated, what the update will be evaluated against, what the rollback criteria are — and the regulator (or notified body) approves the envelope, not each individual update. Inside the envelope, model updates execute under the manufacturer's QMS without a new conformity assessment; outside the envelope, a new assessment is required. This is the AI equivalent of ICH Q12 Established Conditions, and the same pre-emption logic applies: file the envelope you actually need, with the evaluation criteria you can actually defend.

Article 14 requires that high-risk AI systems be designed to be effectively overseen by natural persons during the period of use, with the ability for an overseer to fully understand the system's capacities and limitations, remain aware of automation bias, correctly interpret outputs, decide not to use or override the output, and intervene or interrupt the system. For pharma and medical-device contexts, this maps onto existing human-factors and usability obligations (IEC 62366 for devices; GAMP 5 second edition for system criticality) but adds the explicit automation-bias and interpretability requirements that the existing frameworks under-specify. Article 13 transparency adds an instructions-for-use obligation — the deployer must receive enough information to operate the system safely and to satisfy the deployer's own Article 26 obligations. The recurring failure mode is treating the model as a black box with a UI on top — Article 14 requires the overseer to actually understand the model's limitations, which forces documented capability boundaries, known failure modes and a defined intervention pathway. A QA lead who cannot describe in one paragraph when the model should not be relied on cannot satisfy Article 14.

For a regulated manufacturer, the AI Act does not replace the existing computerised-systems framework — it stacks on top of it. EU GMP Annex 11 (computerised systems) and 21 CFR Part 11 (electronic records and signatures) continue to govern any GxP system using AI; GAMP 5 second edition (July 2022) and FDA CSA (September 2022 draft, finalisation pending) continue to govern the validation approach; ICH Q9(R1) continues to govern the risk methodology. The AI Act adds: a separate risk-tiering exercise (Article 6), data-governance obligations beyond ALCOA+ (Article 10), the technical-documentation pack per Annex IV, automatic logging beyond audit trail (Article 12), the human-oversight specification (Article 14), the post-market monitoring system (Article 72) and, for high-risk systems, the conformity assessment and EU database registration (Articles 43 and 49). The clean operating pattern is one risk-assessment exercise feeding both stacks, one validation pack with the AI-Act-specific evidence added as Annex IV sections, one change-control workflow that recognises both the Annex 11 change category and the AI Act PCCP envelope, and one post-market monitoring system feeding both the GxP CAPA and the AI Act incident-reporting obligations.

High-risk AI systems require a conformity assessment before being placed on the market or put into service. The route depends on the system: Annex I-product safety-component systems (most medical-device AI) follow the conformity-assessment route of the underlying sectoral regulation (MDR/IVDR notified-body involvement), with AI Act conformity evidence integrated into the existing technical documentation. Annex III stand-alone high-risk systems mostly follow an internal control-based conformity assessment (Annex VI), with notified-body involvement only for the biometric-identification subset (Annex VII). The output is an EU declaration of conformity (Article 47) and CE marking (Article 48), plus registration in the EU database for high-risk AI systems (Article 49) before the system is placed on the market — Annex VIII specifies the registration content. For most pharma and medical-device manufacturers the practical path is to integrate the AI Act technical documentation into the existing MDR/IVDR technical file (or, for non-device AI, build the Annex IV pack as a stand-alone artefact) and run a single notified-body engagement rather than two.

Article 72 requires providers of high-risk AI systems to establish and document a post-market monitoring system proportionate to the AI technologies and the risks. Article 73 requires reporting of serious incidents — defined in Article 3(49) as an incident or malfunctioning leading to death, serious harm to health, serious and irreversible disruption of critical infrastructure, breach of fundamental-rights obligations or serious harm to property/environment — to the market surveillance authority of the Member State where the incident occurred. Reporting timeline: immediately after the causal link is established or reasonably likely, and in any event not later than 15 days after the provider becomes aware (10 days for death, 2 days for widespread infringement). For pharma and medical-device manufacturers this overlaps with — but does not replace — MDR Article 87 vigilance reporting, IVDR Article 82, FDA MedWatch / 21 CFR 803 MDR, and PV reporting under Module VI of GVP. The clean operating pattern is one incident-classification workflow that routes each event into every reporting channel it triggers, with timer-managed deadlines per channel.

Chapter V of the AI Act regulates general-purpose AI models (Articles 51–55) and adds an additional tier for GPAI models with systemic risk (broadly, models trained with more than 10^25 floating-point operations, captured under Article 51). Providers of GPAI models have technical-documentation, copyright-policy, training-data-summary and cybersecurity obligations from 2 August 2025. For most regulated manufacturers the practical question is downstream: when a manufacturer integrates a third-party GPAI model (or a foundation-model API) into a high-risk AI system, the upstream provider's obligations do not absolve the downstream manufacturer of its Article 9–22 obligations. The contract with the GPAI provider needs to pass through the data, documentation and incident-cooperation obligations the downstream stack requires; the manufacturer remains the high-risk system provider for the integrated product. The Commission's GPAI Code of Practice (2025) and the AI Office's template documentation give practical text for the contract pass-through.

The Act's phased application matters: prohibited-practice rules from 2 February 2025; GPAI obligations from 2 August 2025; general high-risk and Article 50 transparency obligations from 2 August 2026; high-risk obligations for AI as a safety component of Annex I-regulated products from 2 August 2027. Penalties are severe — up to €35 million or 7% of worldwide annual turnover for prohibited-practice violations, up to €15 million or 3% for non-compliance with most other obligations, and up to €7.5 million or 1.5% for supplying incorrect information to authorities. Realistic 12-month readiness path: Months 1–2: inventory every AI use case in the manufacturer's footprint (in-house, vendor, embedded in equipment), classify each against Articles 5/6/50 and identify the GPAI dependencies. Months 3–5: for the high-risk subset, gap-assess against Articles 9–22 with cross-mapping to existing Annex 11/Part 11/GAMP 5/IEC 62304 evidence — most of the QMS and risk-management obligations are already partly satisfied. Months 6–8: close the Article 10 data-governance gap, draft PCCPs for any system with planned updates, build the Article 14 operator cards and the Article 12 logging extension. Months 9–10: build the Annex IV technical documentation, run the conformity assessment (notified body if required), draft the Article 47 declaration of conformity. Months 11–12: register in the Article 49 database, train deployers on Article 26 obligations, integrate Article 72/73 monitoring and incident reporting into the existing post-market system. Treat the first high-risk system as the template; everything afterwards is delta.