DeepMind Health has been told that unless it changes its public reputation its work could be undermined.
The delayed and much anticipated independent review into Google’s artificial intelligence branch was published on Wednesday.
The independent review panel said that DeepMind’s links to the US search giant mean the company must make extra effort to engage the public in its patient data work.
“As far as we can ascertain, DMH [DeepMind] does not share its data with Google, yet the public perception that this might be the case, now or in the future, will be difficult to overcome and has the potential to delay or undermine work that could be of great potential benefit to patients”, the report stated.
A DeepMind spokeswoman told Digital Health News: “We have always been clear that the patient data we process has never been – and will never be – linked to Google products or services or commercialised in any way”.
Published 5 July, the report was commissioned by DeepMind in February 2016. The partnership between DeepMind and the Royal Free London NHS Foundation Trust to deploy the acute kidney injury alerting app, Streams, was the focus of much of the report.
The review comes just two days after the Information Commission’s Office (ICO) said the initial partnership between the Royal Free and DeepMind, which saw 1.6 million patient’s records transferred, broke data protection law.
Chair, Julian Huppert, said in his foreword that while there are huge benefits for using health data, there are concerns with privacy and confidentiality and “these are magnified if a commercial organisation is involved”.
“‘Good enough’ is not good enough for a company linked so closely to Google, a company that already reaches into every corner of our lives.”
The report said the company’s Google connection means it should be held to higher accountability than other suppliers.
“We believe that it is right that DeepMind Health should be held to higher standards, even if that means they are singled out as a lightning rod for public concerns.”
The panellists criticised DeepMind’s lack of engagement with the public’s concerns over data sharing.
The company, “did not do enough to allay concerns expressed in the media, that patient data would be combined with other data held by Google and used for other purposes”.
In April 2016, a New Scientist investigation exposed the huge quantity of patient data that DeepMind were receiving from the Royal Free, which led to headlines across the national and trade press and an ICO investigation.
The review said that following this press coverage, a new information sharing agreement was done. “This is much improved and more typical of other NHS customer/IT supplier relationships”, said the report.
In response to the panel’s report, DeepMind co-founder, Mustafa Suleyman and clinical lead, Dominic King, released a statement that said they were grateful to the panellists.
“As a result of this process, DeepMind Health has committed to a series of changes to our work and practices to try to set higher standards in our second year. We know we need to work harder to be responsive and accountable to the needs of a far greater cross-section of medicine and society.”
“This includes significantly improving our work with patients and the public, and continuing on the path of greater engagement with Royal Colleges, professional bodies and many other groups in the NHS community.”
In a leaked letter to the media in May, the National Data Guardian, Dame Fiona Caldicott, found that DeepMind had received NHS patient data on an inappropriate legal basis.
The panellists took legal advice on the Royal Free and DeepMind data sharing issue and determined that DeepMind “only acted as a data processor” on behalf of the trust.
“Consequentially, any issues about data protection obligations or confidentiality obligations arising from the use of patient data during testing are in law, matters for the Royal Free as data controller, and we will not comment further on them.”
For all its NHS relationships, the review believes it needs to engage the public audience more.
“It is of particular concern to the Independent Reviewers that DMH’s [DeepMind] public engagement work is less coherent or visible than its work with patients”, and it was recommended that a public engagement strategy was set up.
Suleyman and King added that: “We know that scrutiny can only make our work better and we’re grateful to everyone who has taken time to tell us what we’re getting right, and what we’re getting wrong”.
The panellists have a £50,000 budget from DeepMind, but this year spent £59,315.
DeepMind was founded in 2010, and acquired by Google in 2014. The company began working with the Royal Free in July 2015, and DeepMind Health was founded in February 2016.
Royal Free chose not to comment for the story.
The unpaid review panel are:
- Mike Bracken
- Martin Bromiley
- Elisabeth Buggins
- Eileen Burbidge
- Richard Horton
- Julian Huppert
- Donal O’Donoghue
- Matthew Taylor
- John Tooke