Morley: ‘We need greater regulation of ambient voice technologies’

Self-confessed “100% health nerd” Dr Jessica Morley is a leading voice on ethics in digital healthcare and one of the most exciting speakers on the health tech conference circuit.

Based in the US, where she is a postdoctoral researcher at Yale University Digital Ethics Center, Morley combines ardent enthusiasm for digital with a blunt rejection of hype.

Ahead of speaking at Digital Health Summer Schools, the former tech advisor to the UK government told Digital Health News why she objects to the “hyper-personalisation” of AI in the NHS and how the danger from uncontrolled use of ambient voice technologies is greater than it seems.

You have suggested the NHS is too focused on individual health and has “missed the opportunity to leverage AI for the population”.  What needs to change?

This is basically the continuation of a long running trend to make health less about the system and the state’s responsibility and more about the individual’s responsibility. And that’s couched in terms of ‘empowering’ the person, but there are all these systemic things that impact people’s ability to change their health status.

If you don’t also change the surroundings and make it easier for people to act on information you are essentially just victim blaming.

Doing things that tackle the social determinants of health is very complicated: what are the factors, how are they interacting, and how do we intervene in the system? AI has a phenomenal ability to unpick and help us understand those things.

Ideally, I would like the 10 year health plan, or the things that follow on from it, to not just be about this hyper-personalisation.

A lot of NHS data strategy has been about building vertical data sets, but what we want is more horizontal data sets with the capability to look at things like environmental data.

There are growing numbers of people in the foothills who recognise that [hyper-personalisation] is a problem. But until someone makes a compelling economic argument that focusing only on using AI for individual care is not going to deliver the efficiency gains the government wants, I think we’ll struggle to make the change.

The UK government has published guidance to encourage the use of ambient voice technologies (AVT). Is the focus on AVT necessary?

The guidance is a response to reality. If you don’t use these things in a controlled way, they’re very dangerous.

I think the level of risk posed by these technologies is underestimated. People think that all they’re doing is listening, but they’re not – they’re interpreting and inferring, and we all know that there are problems with hallucinations and accuracy.

Most of them should fall within the specification of software as a medical device and they are not currently recognised as that.

There should be far greater regulation of these tools. There are cases where these things are inventing that someone had a test they are supposed to have – and then it never happens. You start having knock-on effects such as misdiagnosis or late diagnosis.

I don’t think we can blame the market, because the market is responding to what it thinks the system wants.

How do you rate the UK’s chances of becoming, in the prime minister’s words, “an AI superpower”?

It is just political speak – but the UK is by no means alone. That’s what the US and the EU is saying. It’s what China is saying.

In terms of how realistic it is, there are trade-offs between the different big players in this space. There is significantly more money in the US and a big move to deregulate that’s coming from the current administration.

We need to import more people to become a genuine global superpower in AI

We have fantastic universities in the UK, amazing research, and potentially excellent health data. But we don’t have people power in the UK. We need to import more people to become a genuine global superpower in AI, but you can’t do that if you’re also restricting people who are coming in with skilled visa entries.

Sometimes the complexity of what it really takes to become an AI superpower is misunderstood. Lots of complexities were, I think, missed in the AI opportunities plan.

You have people on the ground who are struggling to keep their head above water. And instead of listening to those problems and saying “what do we need to do to fix this?” the government says “AI is the solution – you should just adopt more AI’”.

The reality is there are hospitals that don’t have an electronic health record and places that have one computer between three different wards. Those things need fixing first.

Can we use health data ethically without slowing the rate of progress?

Yes – it is entirely possible to leverage data ethically, acceptably, and not slow the pace of innovation. The idea that either you have the public good or patient privacy is a complete myth.

It is a technical issue that we have largely solved. [The problem is] people are lazy and don’t want to work in more complicated ways. Instead of dealing with the risk technically what they have done is put in place an awful lot of forms. You can reduce the number of forms by building the technical infrastructure – and a better technical infrastructure would be ethical.

You can protect patient privacy, make it far more transparent and accountable, by making sure there are restrictions on who is accessing it, who can see it, and tracking what it is going on.

Can you give us a sense of what you will be speaking about at Summer Schools?    

AI and the NHS – in a nutshell.

Jessica Morley will be speaking at Digital Health’s Summer Schools 2025, 17-18 July at the University of Warwick. The event is supported by Networks sponsors Alcidion, AWS, Altera, Bridgehead, CereCore, Dell Technologies + AMD, Imprivata, Intersystems, Microsoft and Salesforce.