The Information Commissioner’s Office has found a high-profile trial between a London NHS trust and Google’s DeepMind artificial intelligence arm did not comply with the Data Protection Act.
In a keenly anticipated ruling, the ICO announced today that the Royal Free NHS Foundation Trust failed to comply with the Data Protection Act when it provided details on 1.6m patients to Google DeepMind.
The ICO investigation found several shortcomings in how the data was handled, including that patients were not adequately informed that their data would be used as part of the test.
The Information Commissioner observed “the price of innovation does not need to be the erosion of fundamental privacy rights”.
The information exchange saw personal data from about 1.6 million patients transferred to DeepMind to test an alerting system for acute kidney injury, named Streams. A media investigation questioned the scale of the patient data, its use and patient’s knowledge of how their data was being used.
A letter, sent to the Royal Free from the ICO Commissioner, Elizabeth Denham, said that the exchange “did not fully comply with the requirements of the Data Protection Act 1998” and listed a number of “shortcomings” with the data processing deal.
Denham said: “Our investigation found a number of shortcomings in the way patient records were shared for this trial. Patients would not have reasonably expected their information to have been used in this way, and the Trust could and should have been far more transparent with patients as to what was happening.”
Denham added: “The Royal Free did not have a valid basis for satisfying the common law duty of confidence and therefore the processing of that data breached that duty”.
The ICO investigation, begun last May, found that the means to inform patients about how their data was being used were not adequate, and therefore they could not opt-out.
“The evidence presented to date leads the commissioner to conclude that data subjects were not adequately informed that the processing was taking place and that as result, the processing was neither fair nor transparent.”
“Put plainly, if the patients did not know that their information would be used in this way, they could not take steps to object.”
Royal Free has been asked to agree to a set of changes to allow the data sharing to continue.
In her letter, Denham said that the processing of patients records by DeepMind “significantly differs from what data subjects might reasonably have expected to happen to their data when presenting at the Royal Free for treatment”.
The example being a patient presenting to A&E would not expect their data to be accessible to a third party for testing of a new mobile application.
The number of patient records was also found to be “excessive”, unnecessary and out of proportion by the ICO.
DeepMind welcomed the report, but said mistakes were made.
In a blog post on DeepMind’s website, Mustafa Suleyman, co-founder and head of applied AI and Dominic King, clinical lead, said “in our determination to achieve quick impact when this work started in 2015, we underestimated the complexity of the NHS and of the rules around patient data, as well as the potential fears about a well-known tech company working in health.”
“We were almost exclusively focused on building tools that nurses and doctors wanted, and thought of our work as technology for clinicians rather than something that needed to be accountable to and shaped by patients, the public and the NHS as a whole.”
“We got that wrong, and we need to do better.”
Denham said in a statement that: “There’s no doubt the huge potential that creative use of data could have on patient care and clinical improvements, but the price of innovation does not need to be the erosion of fundamental privacy rights.”
She said the ICO has asked the trust to commit to changes to address these concerns, and added that “the Data Protection Act is not a barrier to innovation, but it does need to be considered wherever people’s data is being used”.
To comply with data regulations the trust has been asked by the ICO to:
- Establish a “proper legal basis” under the Data Protection Act for the DeepMind project
- Set out how it will comply with its duty of confidence to patients in any future trial involving personal data
- Complete a privacy impact assessment
- Commission an audit of the trial, the results of which will be shared with the ICO
The shortcomings found by the ICO breached the following data protection principles:
- Principle One: Personal data shall be processed fairly and lawfully
- Principle Three: Personal data should be adequate, relevant and not excessive
- Principle Six: Personal data shall be processed in accordance with the rights of data subjects
- Principle Seven: Appropriate technical and organisational controls shall be taken – this includes the need to ensure that appropriate contractual controls are in place when a data processor is used
The ICO investigation, concerns about how Streams handles patient consent, and disquiet about the large-scale transfer of patient records to a Google company, have not stopped Deepmind signing up other trusts.
Deals are now in place with University College London Hospitals NHS Foundation Trust, Moorfields Eye Hospital NHS Foundation Trust, and Imperial College Healthcare NHS Trust and Taunton and Somerset NHS Trust. Each of the trusts will likely want to review their approach to patient consent and data transfers in light of the ICO’s report on the Royal Free.
The Royal Free said in a statement: “We accept the ICO’s findings and have already made good progress to address the areas where they have concerns. For example, we are now doing much more to keep our patients informed about how their data is used.
“We would like to reassure patients that their information has been in our control at all times and has never been used for anything other than delivering patient care or ensuring their safety.”
In DeepMind’s response, the company also cited how it’s looking to address transparency issues through its independent review panel and a transparency audit.
However, the first report from the independent reviewers has still not been published, with DeepMind saying it will be out “soon”.
Nicola Perrin, head of Understanding Patient Data, praised the ICO’s findings:
“Key lessons – the need for transparency, public engagement and proportionate use of data – must be learnt, so that everyone can have confidence that patient data is being used responsibly. It is good that both DeepMind and Royal Free have recognised that mistakes were made, and are now taking steps to address the concerns. The ICO ruling makes clear that data protection and innovation can work together for the benefit of patients.”