We need to ensure that AI in healthcare is safe, fast and trusted

A new commission will help ensure that the UK’s regulatory system can cope with AI, writes Henrietta Hughes, patient safety commissioner for England 

An incredible wave of scientific and technological change is already taking place in healthcare in the UK with the use of AI in earlier detection of disease, diagnosis and ambient voice technology.

Regulation of medical devices largely predates AI and was designed for stable devices with defined uses which do not change over time. AI health products are being marketed directly to consumers with less professional mediation and there are lower barriers to entry for AI start-ups, with much faster development cycles.

As an NHS GP using AI devices in my clinical practice, I can see the potential benefits that AI can bring, by freeing up clinician time from routine tasks, improving patient experience and potentially safety as well.

Not a panacea

We need to bring patients, the public and the workforce with us so that AI is used to improve patient care and outcomes and empower patients to be true partners in their care. However, we also need to be mindful about the pitfalls of using AI as a panacea.  AI is a tool that comes with potential risks.

With opaque algorithms, product drift, and potential for hallucinations and bias, there are risks that healthcare inequalities will be widened and clinicians deskilled. There is also risk of a loss of the human-to-human interaction which is at the heart of providing healthcare.

Regulation of medical devices largely predates AI and was designed for stable devices with defined uses which do not change over time

The 10 year health plan for England has the ambition to make the NHS the most AI-enabled care system in the world and through this to promote economic growth.

But given the level of uncertainty and the fast pace of change, how can patients and the public know that the use of AI in healthcare is safe, fast and trusted? This depends on the deployment context of AI products.

Developing a regulatory framework which can swiftly bring new devices into the market, is flexible to cope with unknown future changes, and keeps patients and healthcare workers safe, is a global challenge.

Rather than an excessively high barrier to entry, sophisticated post market surveillance is a key element to continuous monitoring of performance in real world situations.

The analogy is a series of hurdles rather than the high jump, which should enable novel devices to reach patients and the public more quickly and ensure confidence in safety at the outset and on an ongoing basis.

Making AI safe and effective

Safe, fast and trusted regulation of AI depends on three key components:  the model, the supplier and the deployer.

The model requires evidence of testing and validation in silico and in real clinical environments. It needs to be clear on intended use and on the data on which it was trained, to control for potential bias.

The supplier needs to provide robust performance management and quality management, with rapid, transparent reporting of safety signals to the regulator, and a responsible approach to model upkeep.

The deployer, who could be a clinician or a healthcare provider, needs to understand how to use AI in clinical workflows, with clear quality assurance processes and a greater emphasis in accrediting institutions for the responsible use of AI in healthcare.

When all three operate as intended, AI is likely to be safe and effective.

This is clearly a highly complex and important issue, with competing views and it is necessary to hear a wide range of perspectives and experiences to ensure that we keep patients and the public at the heart of decision making.

National commission launch

In October 2025, the Medicines and Healthcare products Regulatory Agency (MHRA) launched the National Commission into the Regulation of AI in Healthcare, chaired by Professor Alastair Denniston with myself as deputy chair.

This national commission, formed of global experts and supported by specialist working groups with domain expertise and by the Health Foundation as research partner, will construct an expert and authoritative process for solutions to the challenges set out above.

Over the next year, the national commission will help advise the MHRA on a new regulatory framework for AI as a medical device which will support delivery of the 10 year health plan and its three strategic shifts.

This is key to the MedTech sector harnessing the potential of the UK as a great place to start and grow AI businesses. It is globally relevant and important, and other regulatory jurisdictions are looking with interest at the work in the UK.

For more information visit here. To register your interest in the public call for evidence contact the MHRA at info@mhra.gov.uk