Skip to content
On 19 July 2024, people across the world woke up to find their IT equipment was completely unusable – crashing and unable to restart. An estimated 8.5 million devices were affected during the Crowdstrike outage and what followed was one of the largest outages in the history of Information Technology.
With such a big outage, many industries – including healthcare – were affected and the full impact of the outage is yet to be determined, but estimates of financial damage currently stand at around $10 billion.
Post Incident Review
A preliminary investigation by Crowdstrike has already identified the causes of the catastrophe and what Crowdstrike could have done to avoid it. The summary of the investigation has pointed at a validation failure in release processes; something potentially avoidable with adequate design, development and release controls.
How can healthcare IT manufacturers avoid their own Crowdstrike scenario?
As Healthcare IT becomes increasingly integrated with and central to healthcare operations, software developers and testers should work tirelessly to prevent and detect bugs in their software code, but there are several established International standards which can help to standardise software security.
IEC 62304 –Medical Device Software – Software Life Cycle Processes
IEC 62304 is the international standard for software development and testing of medical device software that is well-recognised as the best approach by many medical device regulators (including the UK MHRA, EU European Commission and US FDA).
IEC 82304-1 – Health Software – General Requirements for Product Safety
IEC 82304-1 is the international standard for product safety of health software and it recognises the development aspects of IEC 62304 as relevant and applicable to all health software development.
Arguably by applying these standards to the development of all health software, alongside strong business continuity and disaster recovery controls, developers can avoid their very own CrowdStrike disaster.
In this blog, we have condensed the requirements of these standards to help businesses understand what preventative action can be taken to safeguard their software and users from harm.
Identifying Common Software Defects
When developing software, it’s common for Engineering teams to identify common ways in which defects can occur and assess risks related to these defects in the code base. Clause 5.1.12 of IEC 62304 encourages development teams to document identified defects and the associated risks whilst documenting how preventive or protective action can prevent issues.
This can be as simple as using an Integrated Development Environment that checks syntax for your code or having a second member of the engineering team review your work prior to release or establishing a User Acceptance Testing environment (UAT). There are also automated tools that can review and/or software code before it is released.
Unit Implementation and Verification
As part of creating software “units” – the smallest pieces of software being created by the developer – Clause 5.5 requires developers to establish a process for checking the software units, setting acceptance criteria for each unit (including management of risks, fault handling, boundary conditions and memory overflows) and to confirm that the software units meet all of the identified acceptance criteria. This was reported in the Crowdstrike incident as one of a number of contributing factors to the disruption:
“…problematic content in Channel File 291 resulted in an out-of-bounds memory read triggering an exception. This unexpected exception could not be gracefully handled, resulting in a Windows operating system crash (BSOD).”
Integration and System Testing
In an environment like Healthcare, Interoperability is critical to improving patient care and clinical workflows. By enabling a complete picture of the patient to be accessible to the healthcare professional, it has been evidenced to improve and enhance the quality and speed of care. Many healthcare technology businesses work in relative harmony with other software, meaning a robust approach to integration testing is critical.
Integration Testing (Clause 5.6) involves confirming the interfaces between software units within the software product and interfaces with other software function correctly. System Testing (Clause 5.7) confirms that the finished software meets the customers requirements and the intentions of the developers. By testing software in environments that are similar to their customers, such as UAT environment that mirrors a Live production environment, developers identify bugs or issues that may not occur during development testing and can take corrective action before causing harm.
Software Release
Clause 5.8 requires that prior to Software Release a review of the design, development and testing work is undertaken to ensure they are sufficient, proportionate to the risk of the release and they have been completed. . By having clear lines of approval documented within a Standard Operating Procedure, developers can identify and resolve more issues prior to them reaching customers and affecting patients.
In much the same way as Clauses 5.6 and 5.7 require customer approval and quality testing, Clause 5.8 exists to ensure a robust Software Release process is in place that can further screen issues prior to release.
One way to mitigate significant risks during software release is to have a staggered deployment. This reduces the chance of widespread issues with only a small subset of customers being affected and the ability to roll-back the release should the issues be significant. Whilst this may take more time, in a healthcare setting, it’s a sensible approach to ensure the safety of the software.
Product Risk Management
Under Clause 4.2, developers are required to implement a risk management system for their product – managing risk throughout the product life cycle. For Medical Devices, this is addressed under ISO 14971 for Medical Device Risk Management.
In addition to the international standards, at a national level in the UK, the adherence to the DCB0129 Clinical Safety standard enables the assessment of specific practices that are referenced above, many of which stem from ISO 14971. You can learn more about the crossover in a recent blog here.
By applying the requirements of DCB0129, health software developers can be assured that a registered healthcare professional has not only risk assessed and approved the technology itself, but also reviewed existing policies and procedures mitigating the product’s risks. This includes a review of testing strategies, outstanding bugs, backup and restore policies and downtime procedures. Should a health software developer be in the early stages of the software development process, then a Clinical Safety Officer can support them to create and implement strong policies and procedures to support the use of their technology.
An independent, clinician-centric voice in the development process can be essential in identifying issues that only a clinician would see – we’ve certainly found this when supporting clients with DCB0129 and the NHS agree; compliance with DCB0129 is mandatory when selling into publicly-funded health and social care in England.
Would implementing IEC 62304 have prevented the CrowdStrike disaster?
It’s difficult to say as we only have the preliminary report and little in-depth knowledge of their company’s workings.
However, IEC 62304 is recognised as best practice for health and medical device software and following its principles, alongside other standards such as DCB 0129, will certainly reduce the probability of a software developer experiencing an issue on this scale.
How we can help
At 8fold, we work with digital health and medical device software companies to support them in developing software that is safe, secure and helps patients by providing the evidence and support to prove it to their customers and the regulators. This includes supporting the implementation of a Management System to meet the requirements of IEC 62304 (as well as other internationally-recognised medical device standards key to software medical devices – ISO 13485 Medical Device Quality Management, ISO 14971 Medical Device Risk Management and IEC 62366 Medical Device Usability Engineering).
Contact us to find out more about how we can help you.
Book your free, no-obligation discovery call with our experts.
Book your free, no-obligation discovery call with our experts. If you need support with NHS compliance, international standards or regulations, we’ve got your back. Get in touch.
