What is a Clinical Evaluation?
Worldwide Medical Device Regulations establish a requirement to perform “Clinical Evaluation” on any medical device, but what is a Clinical Evaluation and how can it be performed on a SaMD?
EU Medical Device Regulations define a Clinical Evaluation as:
“a systematic and planned process to continuously generate, collect, analyse and assess the clinical data pertaining to a device in order to verify the safety and performance, including clinical benefits, of the device when used as intended by the manufacturer”
But we don’t think this is the most helpful definition without further guidance. Clinical Evaluations are ultimately the way that regulators have set up and agreed to make manufacturers prove their medical claims about their products.
In this blog, our very own Daniel Mannion shares his experience of working with SaMDs and performing Clinical Evaluations that meet regulatory expectations without bankrupting your company.
Why do we do Clinical Evaluations?
In the healthcare industry, the Ronseal Test applies – it has to do exactly what it says on the tin (or label). Unlike most consumer products, the level of proof expected when it comes to healthcare is much higher because of the higher risks (in general) presented by medical devices compared to other products.
Which Medical Devices need Clinical Evaluations?
All medical devices require a Clinical Evaluation – in the UK, Annex I of the UK Medical Device Regulations* establishes the need to perform a Clinical Evaluation for all medical devices. In the EU, Article 5 and Article 10 require manufacturers to perform a Clinical Evaluation. In the US, the FDA doesn’t necessarily require something called a Clinical Evaluation, but it does require data on clinical and pre-clinical studies for many medical device types (dependent on device classification).
*Before anyone writes to me – technically, it’s Annex I of EU MDD 93/42/EEC which is adopted through UK MDR 2002 into UK law.
What are the Stages of Clinical Evaluation?
- Scope – Define the scope of the device – the technology in use, the intended purpose, and what claims you want to make about the device.
- Plan – Create protocols and plans about how you will gather and analyse the data needed to support the claims in the context of the technology and the intended purpose.
- Gather – Obtain the relevant data from testing of the device.
- Analyse – Perform the statistical analyses on the data to come to conclusions about the performance of the device.
- Document – Create the Clinical Evaluation Report to bring together all of the data, analysis and conclusions about the device and determine whether the device performs well enough and is safe enough to release.
- Repeat – Clinical Evaluation is an ongoing process – you must continue to gather, analyse and document data to prove that the device continues to be as safe and effective as it needs to be. This is known as “Post-Market Surveillance” and “Post-Market Clinical Follow-Up”.
What needs to be in a Clinical Evaluation plan
The exact details of the Clinical Evaluation will vary from device to device, but in general the below are a good starting point:
Reviewing scientific literature
Clinical Investigations (see below)
Identification of the applicable performance characteristics of the device (from the list in the relevant regulations)
Intended Purpose, Patients, Use Environment
Risk Management
Claims to be made about the device
Need help generating your
Clinical Evaluation Plan or Report?
What is a Clinical Investigation?
A Clinical Investigation is a term used in the UK and EU – it’s the part of a Clinical Evaluation that involves real patients – it’s the equivalent of the term “clinical trial” for a medicine, but without the clearly defined “stages”. Regulators pay particular attention to Clinical Investigations both because of the value these studies generate and because using an “unproven” medical device on real patients can be the most risky part of the lifecycle of a medical device.
Not all medical devices require a Clinical Investigation as part of their Clinical Evaluation – clinical data can also be obtained from equivalent devices or from devices released onto a different market (e.g. using clinical data from a device used in the UK to support an EU clinical evaluation), but the relevance of this data needs to be established.
What does Equivalence mean? Why is it relevant?
Within a Clinical Evaluation, clinical data can be obtained and analysed from the medical device your company makes or from a medical device made by another company if the device is very close in design and use to yours. This can be very valuable as it can avoid costly clinical investigations.
For medical devices in the UK, this concept is governed by MEDDEV guidance and in the EU by MDCG guidance, but in principle, there are three areas to consider when demonstrating equivalence:
Clinical
Are the medical devices used for the same thing, same patients, same performance?
Technical
Are the medical devices similar in design, for use in the same environment, have similar physical and chemical properties and principles of operation?
Biological
Do the medical devices contain the same materials that are in contact with the same human tissue(s) (biocompatibility).
Biological characteristics are not really relevant to SaMD, but clinical and technical certainly are and it’s especially worth noting that the EU guidance specifically calls out “similarity” in “software algorithm” as a requirement for equivalence.
What is a Clinical Development Plan?
Clinical Development Plan is specific to EU regulations – it is a plan detailing the series and progression of any clinical investigations undertaken as part of the Clinical Evaluation.
How can this be adapted to SaMDs?
The EU Medical Device Coordination Group (MDCG) has a very helpful framework for considering the Clinical Evaluation of SaMDs. It establishes all software (which includes AIaMD as well) as a tool that takes inputs and converts them into outputs. In their guidance, they establish three aspects of Clinical Evaluation of SaMDs based on this inputs-and-outputs model that can form the basis of the structure of the Clinical Evaluation.
Clinical Association
How does your software’s output associate with the medical condition in your intended purpose? How do you evidence that the output associates with the condition?
Technical Performance
How reliably, consistently and accurately does your software convert inputs into the correct outputs? Is the algorithm driving your software generalisable to the entire intended population? Critically, this includes concepts like reliability (as per IEC 62304) and Usability (as per IEC 62366).
Clinical Performance
How do you evidence that the device provides the intended benefits (or claims) to all of the populations and in all of the environments you have claimed to be able to address?
Evidence for these three areas can be drawn from scientific literature, non-clinical studies (e.g. software verification testing), clinical studies (whether retrospective or prospective) and equivalent devices.
How does the Clinical Evaluation interact with the Technical Documentation (Tech File)?
The Clinical Evaluation Plan, Clinical Development Plan, Clinical Evaluation Report, Characteristics Checklist and any other plans, protocols, reports, datasets etc. all form a big part (although not all of) the tech file for a medical device – in fact, a review by a Notified/Approved Body contains a specific section just to cover the Clinical Evaluation.
How does the Clinical Evaluation interact with the Quality Management System (QMS)?
Your QMS will need to:
- record all of the documents generated in Clinical Evaluation.
- contain a procedure that addresses Clinical Evaluation and Post-Market Surveillance, plus how they interact with complaint handling, incident reporting, risk management and development processes.
- record evidence of the competence of staff involved in the Clinical Evaluation.
- ensure tracking of post-market surveillance and clinical follow-up activities and their impact on the Clinical Evaluation.
- trigger updates to the Clinical Evaluation and even repeated or updated activities if the product changes.
What about IVDs?
In-vitro diagnostic medical devices (IVDs) sit under their own section in the UK medical device regulations, their own regulation in the EU (EU IVDR 2017/746), but are treated no differently in the US.
The equivalent term for an IVD is “Performance Evaluation”; it is similar to Clinical Evaluation for SaMDs – in fact, the MDCG guidance above applies to SaMDs and IVDs due to their similarity.
Do the proposed changes to the UK Medical Device Regulations affect Clinical Evaluation?
There are eight total mentions of Clinical Evaluation in the proposed changes to the UK Medical Device Regulations, with the overall approach appearing to be closer alignment with the EU Medical Device Regulations, including, for example, specific mention of software algorithms as a technical characteristic to be considered for equivalence.
Registering as a Medical Device?
Need help developing a QMS, developing a clear Technical File and providing evidence in your Clinical Evaluation?
Want to learn from decades of experience to accelerate your route to market?
