Skip to content
There is cautious optimism in the medical field about the potential impact Artificial Intelligence (AI) or Machine Learning (ML) could have on patient care in the future, with regulators and governments looking for the best way to mitigate risk without stifling innovation.
Clinical Risk Management has been a cornerstone of healthcare compliance in the UK for a number of years through DCB0129 & DCB0160. With the introduction of AMLAS, there is the opportunity for greater clinician assurance to embrace new technologies without some of the implicit risks often associated with machine learning or AI tools in healthcare.
In this blog, we will unpack AMLAS, the new best practice framework in Clinical Risk Management, and explore how clinicians can feel confident in their use of AI and Machine Learning with clinical risk management controls in place.
What is AMLAS? (Assurance of Machine Learning for use in Autonomous Systems)
Developed by researchers from the University of York as part of the Assuring Autonomy International Programme, AMLAS Safe Machine Learning addresses the unique challenges posed by ML technologies in safety-critical applications. By design, it is aligned with the NHS Clinical Safety Standards, DCB0129 and DCB0160.
Summarised below are key elements of the AMLAS framework and how they can help stop HCPs (healthcare professionals) from feeling like a liability when using ML products:
1.Explainability
AMLAS emphasises the importance of transparency and interpretability in machine learning models. By ensuring that the algorithms’ decisions are explainable, healthcare professionals can understand why certain recommendations or predictions are made, reducing the feeling of being out of control or in the dark.
An example of this may be an AI analysis tool, be it radiology or medication related. The tool may provide a different treatment plan to that which the patient may be already on. The ability of the AI to explain its decision can help the HCP make a clinical decision as to whether to accept or reject the proposed plan.
primary aim of the MLSafety Assurance Scoping argument is to explain and justify the essential relationship between, on the one hand, the system‐level safety requirements and associated hazards and risks, and on the other hand, the ML‐specific safety requirements and associated ML performance and failure conditions
G1.1 AMLAS framework
2.Human-in-the-loop
The framework advocates for a human-in-the-loop approach, where healthcare professionals remain actively involved in decision-making processes. They are not replaced by algorithms but rather collaborate with them. This helps them feel empowered and integral to the process rather than being sidelined.
Tools can be developed with functionality to allow HCPs to document the final clinical decisions they have made to a patient’s care. This empowers HCPs to continue clinical decision making with documentation to advocate for their practice.
“For some safety‐related properties, such as interpretability, it may be necessary to include a human in the loop evaluation mechanism. This may involve placing the component into the application and generating explanations for experts to evaluate” Note 34 AMLAS framework
3.User-centred design
The framework emphasises user-centred design principles, meaning that the software is designed with the needs and workflows of healthcare professionals in mind. This can help them feel more comfortable and confident in using the technology, as it fits seamlessly into their existing practices rather than adding complexity or burden.
The importance of user-centred design is also highlighted by the Digital Technology Assessment Criteria (DTAC), the national baseline for compliance for health and social care in the UK. Both user-centred design and accessibility are about making tools and services easier to use by the majority of the population. Whilst usability and accessibility are often overlooked, illustrated by the non-mandatory expectation for compliance set out in the DTAC, it can sometimes be the difference between the successful adoption and implementation of digital health solutions.
Artefact [EE]: Operational Scenarios: HCP can contribute towards describing ‘operational scenarios’ which translates to specific healthcare use cases.
“The set of operational scenarios shall therefore represent real scenarios that may be encountered when the system is in operation. This set shall comprise a number of defined scenarios, meaningful with respect to the safety requirements of the system, that may occur during the system life”
4.Ethical considerations
AMLAS incorporates ethical considerations into the development and deployment of machine learning systems. By ensuring that the software adheres to ethical standards, such as patient privacy and fairness, healthcare professionals can trust that their use of the technology aligns with their values and professional standards.
Bad, sensationalised or even fake news pertaining to the rapid development of AI can cause an air of scepticism and avoidance within HCPs. Healthcare professionals are held accountable to their regulatory body and if manufacturers can provide evidence and reassurance that AI is adhering to global or national standards, then both trust of the technology and use of it, are likely to be higher.
“The views of stakeholders, including domain experts and users, who have a deep understanding of the context into which the system is to be deployed. These opinions may be founded on:
– A scientific understanding of the processes at play e.g. vehicle dynamics.
– An understanding of the legal and ethical frameworks which govern the context.
– Personal experience.
– A study of similar systems and the lessons learnt from failures within these systems and operational contexts”
AMLAS Framework Artefact A System security requirements
Overall, the AMLAS framework aims to create a supportive structure where healthcare professionals can embrace machine learning software as a valuable tool rather than viewing it as a threat to their expertise or liability.
How does AMLAS work alongside DCB0129 / DCB0160?
DCB0129 and DCB0160 are standards that have been designed by NHS Digital to assist manufacturers and providers of healthcare software. The standard provides a structured approach to ensure clinical risk is managed appropriately by organisations that are responsible for the development, maintenance and implementation of health IT systems.
A longstanding issue of technology being adopted by HCPs in the NHS, has been the hindrance vs helpfulness debate one faces when time & resources cause daily pressures within the healthcare setting. This is further complicated by a lack of knowledge and transparency of AI products.
AMLAS provides the opportunity for a further structured approach with the machine learning element of health IT systems. The opportunity for manufacturers to provide oversight on key functionalities and workings, not only empowers the NHS when procuring such a system, but also enables risks associated with adoption, use and requirements to be mitigated and thought about, prior to any use of the system.
How does DCB0129/DCB0160 and AMLAS help clinicians embrace AI?
Working closely with clinicians, 8foldGovernance has come across many expressions of concern about the amount of liability clinicians risk taking through using digital health devices and software, especially where AI is involved.
The Director of the Josiah Macy Foundation (which focuses on medical education) said in a recent interview, “AI is not perfect; and, in malpractice, you want to sue a human.”
Therefore it is the human at the end of the healthcare decision-making process (the nurse, doctor, midwife etc.) who will require confidence in the AI tool. This can only be achieved when transparent & structured proof is made available about the AI, its data and explainability. Currently, DCB0129 and DCB0160 can provide a glimpse of the structure required around AI products, especially pertaining to HCP’s trust. The AMLAS standard can offer additional reassurance and help alleviate concerns when integrating AI products in healthcare by providing clear guidelines and best practices tailored to the specific challenges and risks associated with AI implementation.
Further reading:
Clinicians Risk Becoming “Liability Sinks” for Artificial Intelligence T. Lawton et al 2024
