AI project to predict health outcomes paused over GP data concern

NHS England has paused development of a generative AI model capable of predicting future health outcomes after doctors raised concerns about the use of GP patient data.

The model, called Foresight, trained on de-identified NHS data from 57 million people in England, was being tested by researchers at University College London and King’s College London as part of a national pilot exploring how AI could support earlier intervention and more personalised care.

However the British Medical Association (BMA) and the Royal College of General Practitioners joint GP IT committee said that it had not been made aware that GP data collected for Covid-19 research had been used to train the AI model.

Dr David Wrigley, BMA England GP committee deputy chair, said: “For GPs, our focus is always on maintaining our patients’ trust in how their confidential data is handled.

“We were not aware that GP data, collected for Covid-19 research, was being used to train an AI model, Foresight.

“As such, we are unclear as to whether the correct processes were followed to ensure that data was shared in line with patients’ expectations and established governance processes.

“We have raised our concerns with NHS England through the joint GP IT committee and appreciate their verbal commitment to improve on these processes going forward.

“The committee has asked NHS England to refer itself to the Information Commissioner so the full circumstances can be understood, and to pause ongoing processing of data in this model, as a precaution, while the facts can be established.”

Wrigley added that patients “shouldn’t have to worry that what they tell their GP will get fed to AI models without the full range of safeguards in place to dictate how that data is shared”.

An NHSE spokesperson told Digital Health News: “Maintaining patient privacy is central to this project and we are grateful to the joint GP IT committee for raising its concerns and meeting with us to discuss the strict governance and controls in place to ensure patients’ data remains secure.

“The committee has asked us to confirm that GDPR [General Data Protection Regulation] principles are being upheld at all times and we have agreed to pause the project while our data protection officer now conducts a review and recommends whether any further action is needed.”

They added that the project has been conducted under a data sharing agreement between NHSE and a consortium of researchers, brought together by the British Heart Foundation Data Science Centre and this agreement was reviewed by the GP Profession Advisory Group.

Also they said that the research has been conducted by approved researchers within NHSE’s Secure Data Environment (SDE) which cannot be accessed from outside that environment.

Results of the research can only leave the SDE after being checked by a member of NHS staff to ensure that they could not be used to identify a patient, NHSE said.