Skip to content
Artificial intelligence is increasingly part of the conversation about very real pressures in the NHS, from workforce shortages to rising demand and constrained resources. If you’re building, deploying, or even just exploring AI-enabled tools in healthcare, that creates both opportunity and uncertainty.
Alongside the promise, you’ve probably got a familiar set of questions:
How will AI be regulated?
Will it be treated as something fundamentally new?
And are we heading towards a wave of additional requirements that make it harder, or slower, to bring products into use?
And if you’re trying to plan product timelines and evidence requirements, those questions aren’t academic. These questions feel particularly live because the Medicines and Healthcare products Regulatory Agency (MHRA) has opened a call for evidence on AI in medical devices. In some parts of the debate, it’s been framed as a signal that AI demands significantly more regulation in order to ensure patient safety.
That framing misses something important. Medical device regulation has always had to balance protection with access. Over-regulation is also a patient safety issue, by delaying beneficial technologies, increasing system burden, or discouraging innovation in areas where it is most needed. AI doesn’t change that underlying reality.
As part of this process, we’ve submitted our views to the MHRA, drawing on experience supporting teams to navigate medical device regulation and governance while deploying AI-enabled technologies in real healthcare settings. We’re less interested in abstract policy debates and more in what works in practice.
In a nutshell: AI does not require a wholesale rewrite of medical device regulation. In this article, we’ll set out how existing frameworks already address many AI-related concerns, where clearer guidance would genuinely help, and what the current conversation means for you as an organisation building responsibly in this space.
If you’re wondering why AI doesn’t require a fundamental rethink of medical device regulation, it helps to start with what regulation is designed to do.
Like most consumer safety regimes, medical device regulation focuses on two core concerns: preventing fraud and protecting patient safety. But unlike many other regulatory systems, it has always had to deal with an additional reality. Patients can be harmed not only by unsafe devices but also by the absence of effective ones.
This is where the idea that over-regulation is also a patient safety issue comes from. Disproportionate requirements can slow access to beneficial technologies, reduce choice, and discourage innovation in areas of genuine clinical need. If you are developing or deploying a medical device, these trade-offs are not abstract. They shape whether products reach patients at all.
This balance between protection and access is already built into how medical devices are regulated in practice. We know this because if regulation were solely about minimising risk, every device would be treated as high risk and subject to the same level of scrutiny as today’s Class III devices. Instead, risk classification exists to allow proportionate oversight, increasing regulatory burden where potential harm is greater and reducing it where risks are lower or better understood.
The consequences of getting this balance wrong are not theoretical. Recent data from the EU indicates that 48 percent of manufacturers have stopped supplying their products to the EU market due to the cost of certifying or re-certifying them under the EU MDR. For teams like yours, that kind of withdrawal is not just a market issue, it directly affects patient access to effective care.
This way of thinking matters for AI. The real question isn’t whether AI is novel or complex. It’s whether it changes the underlying regulatory problem. In most cases, it does not.
Much of the regulatory work needed to govern AI in medical devices already exists, and as we will see, the challenge is less about inventing new rules than about applying established principles thoughtfully and proportionately.
Ai isn’t special: most of the rules already exist.
AI often feels fundamentally different from other types of medical technology. It evolves quickly, can be opaque, and behaves in ways that feel harder to predict than traditional software. If you are building or deploying AI-enabled medical devices, it’s reasonable to wonder whether existing regulatory approaches are really sufficient.
In practice, a lot of the regulatory groundwork is already there.
Over the past few decades, regulators have built robust frameworks for software as a medical device (SaMD) and software in a medical device (SiMD). Successive waves of technological change, including cloud computing, remote updates, and smartphone-based applications, have not required a fundamental shift in regulatory principles. With the exception of continuously learning AI (which we address later), those same principles continue to apply to AI-enabled medical devices when supported by appropriate good practice guidance. In fact, these principles have been applied successfully for years in medical AI applications for radiology imaging, digital pathology and ocular imaging.
What stays the same?
Here are the parts you can treat as familiar. Many of the concerns raised about AI are, at their core, questions you already answer for other software-based devices. The tooling might change quickly, but the governance expectations don’t.
What do you still have to prove?
Existing regulatory requirements distinguish between design verification (confirming that a device has been built as intended) and design validation (confirming that it performs safely and effectively for its intended clinical use). These principles continue to apply to AI-enabled systems, even where components such as large language models (LLMs) may change frequently.
Recent guidance from Notified Bodies shows that these requirements can be applied pragmatically, including for systems that incorporate rapidly evolving consumer AI tools. The obligation remains the same as it is for other software-based devices: to demonstrate that changes are understood, controlled, and do not undermine safety or performance.
What about AI in your QMS?
Concerns are often raised about the use of AI within quality management systems, particularly where AI is used to generate or support regulated documentation. In practice, these are not entirely new governance challenges. Most teams already deal with fast-moving software in their quality environment, from documentation platforms to ticketing and collaboration tools. You still need to show that changes are controlled, people are competent to use the tools, and outputs are appropriately reviewed.
The same types of risk already exist when individuals approve documents without adequate review or rely on software tools without fully understanding their limitations. Experienced quality and regulatory teams already know how to manage these issues within existing technical file structures and quality management systems. For organisations like yours, this is less about inventing new controls and more about applying existing ones consistently and transparently.
How do guardrails and transparency fit in?
Clarity around intended purpose has long been a cornerstone of medical device regulation, and it is especially important for AI-enabled devices. Defining what a device is intended to do, and what it is not intended to do, allows products to be classified appropriately and helps prevent unsafe or unintended use within existing risk classification frameworks.
Many of the transparency concerns associated with AI are already addressed through established labelling and usability requirements. Manufacturers are required to be clear about risks, limitations, and appropriate use, and to provide evidence that users can understand and act on that information. These mechanisms are already designed to support safe reliance on medical devices, including those that incorporate AI, and are more effectively strengthened through guidance than through duplicative new rules.
Put simply, there’s a consistent principle running through all of this:
From a regulatory standpoint, AI is best understood as a method, not the object of regulation.
As with other software-based medical devices, the focus should remain on the quality, safety, and effectiveness of the outputs, rather than on the specific technical approach used to generate them. Maintaining this focus allows regulatory frameworks to remain flexible as AI technologies evolve, rather than becoming tied to today’s particular implementations.
Proportionate evolution, not exceptionalism.
Proposals such as pre-determined change control plans (PCCPs) are often highlighted as particularly valuable for AI-enabled medical devices. While these approaches may be especially helpful in AI contexts, the justifications for them are not unique to AI.
In many cases, PCCPs are sensible evolutions of existing regulatory practice that could benefit a broader range of software-based medical devices. Seen this way, they support proportionate adaptation of the regulatory system rather than the creation of AI-specific exceptions.
Much of what is needed to govern AI safely within medical devices already exists. The challenge is not that AI breaks regulatory principles, but that those principles need to be applied clearly, proportionately, and with appropriate supporting guidance.
That said, there are areas where AI does introduce genuinely different considerations.
Here are the parts you can treat as familiar. Many of the concerns raised about AI are, at their core, questions you already answer for other software-based devices. The tooling might change quickly, but the governance expectations don’t.
Data Management
AI-enabled medical devices depend heavily on the data they are trained, tested, and validated on. For those responsible for bringing these systems into use, data governance is a core safety concern. Practical questions include, for example:
- How do you know your training, testing, and validation data are representative of your intended patient population?
- How do you know those datasets are genuinely independent?
- How do you know the data has not been tampered with, deliberately or accidentally?
Without the right controls, weaknesses in data management can lead to biased outputs, degraded performance, or unsafe results in real-world use. These risks translate directly into patient harm and must be addressed by those developing and deploying AI-enabled devices.
What do clearer standards and recognised frameworks enable for AI?
Clear, proportionate expectations around data governance make these risks easier to manage responsibly. While AI-specific standards continue to emerge, ISO/IEC 42001 already gives you an accessible way to establish an AI management system that integrates well with existing quality frameworks such as ISO 13485, supporting robust and auditable data controls without requiring entirely new governance structures.
AI-enabled systems can be particularly sensitive to changes in the environments they operate in. Imagine you’d proven an algorithm that distinguishes a cold from the flu was performing well in October 2019. How confident would you be using it unchanged in April 2020? That’s model drift in plain terms: the world changes, and performance can change with it.
This risk isn’t unique to AI, but it’s amplified where systems rely on datasets or signals that can shift quickly, such as external databases, population characteristics, or evolving clinical practice.
How existing requirements can be applied effectively.
Existing post-market surveillance requirements already give you a foundation for managing this risk, but clearer guidance on how to adapt them for AI-enabled devices would help. For example, expectations around ongoing review of specialist literature, databases, and real-world performance data can support earlier detection of drift and emerging issues without imposing disproportionate new obligations.
Risk Management for Machine Learning systems
Developing software using machine learning introduces different risks compared to traditional software design. These include poor data quality, bias, and the risk of over- or under-trust by users, all of which can affect safety and effectiveness if not addressed explicitly.
These risks are well understood in principle, but they require careful, systematic treatment to ensure AI-enabled medical devices remain safe in practice.
Where established guidance already supports good practice.
Guidance such as BS/AAMI 34971:2023 provides practical direction on applying established risk management principles to machine learning systems. That matters because it gives you a clearer, more consistent “what good looks like” benchmark and reduces surprises during conformity assessment. Wider recognition and designation of this guidance would support consistent, high-quality risk management for AI-enabled medical devices without creating parallel regulatory regimes.
Continuously Learning AI Systems
Continuously learning AI challenges a core assumption of existing design controls, namely that changes to a medical device can be verified before deployment. Where systems adapt over time, this assumption no longer holds in the same way, and the associated risks matter. It’s also an area the MHRA is already actively exploring through work such as Project Ship of Theseus, which looks at how adaptive systems can be assured over time.
This challenge is not insurmountable, but it does require careful handling. Analogies from physical device design, such as operating within defined tolerances, can help frame how adaptive behaviour might be assessed safely.
Where new mechanisms are needed to manage change safely.
To deploy adaptive systems responsibly, you need a clear way to manage change. That usually means guardrails for learning, staged evaluation of adaptive performance, strengthened post-market surveillance, and more frequent post-market clinical follow-up where risk justifies it.
Liability
People often ask how liability should be allocated where AI systems are involved in clinical decision-making. While AI may change how decisions are informed, it does not fundamentally alter established principles of responsibility.
Manufacturers remain responsible for producing safe and effective devices, including the design of user interfaces. Clinicians are responsible for using devices appropriately and within their intended purpose. Patients are expected to provide relevant information. Shifting this balance would represent a broader change in consumer protection law, which would have much wider ramifications than just in healthcare.
Why existing responsibility frameworks remain fit for purpose.
Maintaining clear, consistent expectations within existing regulatory frameworks helps avoid confusion about roles and responsibilities. Where further clarification is needed, it is better addressed through guidance and interpretation rather than by redefining liability in ways that could create uncertainty or unintended consequences.
Environmental Considerations
The environmental impact of AI-enabled medical devices is a growing concern, particularly given the energy demands of some AI systems and the NHS’s net-zero commitments. While environmental risk is already captured within ISO 14971 and related requirements, AI can make these considerations harder to assess in practice.
A lack of transparency around the infrastructure underpinning AI systems can make it difficult for organisations to understand and manage their environmental footprint.
Where clearer guidance would support consistent, responsible practice.
Guidance on when AI use represents a net environmental benefit, and how carbon impacts should be estimated in an industry that is notoriously opaque on its environmental impact, would help organisations innovate in ways that align with wider sustainability goals.
These examples illustrate where AI introduces practical challenges that organisations must manage in real deployments, and where clearer regulatory support would make responsible practice easier. It is considerations like these that have shaped the specific recommendations we have put forward to the MHRA and NHS.
What we’d like the MHRA and NHS to do.
Below are the recommendations we submitted to the MHRA as part of the call for evidence. They’re based on what we see when we support teams deploying AI-enabled medical devices in real healthcare settings: where existing frameworks already work well, and where clearer guidance or a targeted mechanism would make responsible innovation easier.
Provide clearer, practical guidance for AI-enabled devices
- Release dedicated AIaMD guidance, broadly aligned with the existing Digital Mental Health Technology (DMHT) guidance, to clearly confirm regulatory expectations rather than relying on informal indications of current thinking.
- Publish specific guidance on post-market surveillance for AI-enabled medical devices (or Medical Devices with High Functionality), clarifying how existing PMS requirements should be applied in practice.
- Avoid incorporating a formal definition of Artificial Intelligence into new pre-market regulations, as any such definition is likely to be vague, quickly outdated, and of limited practical value.
Strengthen the use of recognised standards
- Designate BS/AAMI 34971:2023 to support consistent and effective risk management for AI systems incorporating machine learning.
- Designate ISO/IEC 42001 and include it within quality management system audits for higher-risk AI-enabled medical devices, supporting robust and auditable AI governance.
Enable proportionate approaches to change and market access
- Include provisions in new pre-market regulations that allow the MHRA to define mechanisms for the controlled release of adaptive AI systems in future, without requiring further legislative amendments.
- Create a standardised mechanism for evidencing the applicability of AI medical devices to the UK market, supporting planned reliance on approvals from other jurisdictions and reducing unnecessary duplication.
Improve regulatory coherence and efficiency
- Move past the EU-centric baggage of the Approved Bodies system and bring medical device approvals fully in-house within the MHRA, consolidating AI expertise and removing the inefficiencies created by the current Approved Bodies model.
Staying ahead of regulatory change.
The MHRA’s call for evidence shows a regulatory system paying close attention to how AI is being used in practice. While change is likely, it is unlikely to be wholesale. In other words, you don’t need to panic about a sudden overhaul. The system already covers a lot of what matters, and where it needs to adapt, it can do so in a targeted way.
At 8fold, we stay close to this conversation because it directly affects the organisations we support and the patients their technologies are designed to serve. Our work involves helping teams interpret regulations and apply them as it stands today, while anticipating how expectations may shift tomorrow. That combination of applied experience and forward-looking engagement is what shapes our approach.
This is also a moment of opportunity. Through processes like the MHRA’s call for evidence, and alongside wider discussions involving the AI Commission, there is still space for informed, practical perspectives to shape what comes next. If you are building or deploying AI-enabled medical devices, staying engaged and having your say now can make a real difference.
If you’d like support navigating this in a way that’s proportionate and workable, we can talk it through with you.
We’re the experts that have your back.
Speak to a member of our team about how we can help you build “Good Governance” into your organisation to accelerate commercial success
