Digital health AI chatbot firm, Babylon Health, has hit back at a noted Twitter critic who has persistently raised concerns about the company’s symptom triage chatbot service, describing him as a troll.
Dr David Watkins, who goes by the Twitter handle @DrMurphy11, has long been a thorn in the side of Babylon, posting his detailed analysis of flaws revealed from using Babylon’s artificial intelligence (AI) tool based on hypothetical situations.
Dr Watkins said he had first alerted Babylon to the problem with their algorithm three years ago.
Earlier this week, Dr Watkins had unmasked himself at a Royal Society of Medicine event on AI and digital health, where he presented on ‘Cowboys and Unicorns: why validation is more important than valuation’.
During his presentation, he suggested there were “still fundamental flaws” in the chatbot.
He added: “Moving forward, my plea is to have less focus on the prancing unicorns and more on the data to show us that we can ensure that these technologies are accurate, safe, reliable and can be trusted.”
During a discussion afterwards, he clashed with Dr Keith Grimes, clinical artificial intelligence and innovation director at Babylon, who was booed by sections of the audience.
Dr Grimes responded to Dr Watkins claims by saying he has seen “nothing but hard work and dedication” from Babylon staff, and that the findings were “old news”. Dr Grimes also said he was “keen to hear criticism”.
Speaking to Digital Health News, Dr Watkins said he was “truly concerned about patient safety” and has been left feeling frustrated by the “lack of action being taken” by regulatory bodies such as the Medicines and Healthcare products Regulatory Agency (MHRA) and the Care Quality Commission (CQC).
Dr Watkins added that he “has always been willing to engage with the regulators” and “would like to see a more level playing field and open, evidenced based, debate surrounding the use of technology in health care”.
In a press statement, Babylon refers to Dr Watkins as a “troll”, claiming that he’s “targeted members of our staff, partners, clients, regulators and journalists and tweeted defamatory content about us”.
The statement goes onto say that the track record of Babylon’s AI “speaks for itself” and had been “used millions of times” with “not one single patient” reporting any harm.
Babylon has a track record of seeking to silence perceived critics, having previously taken legal action against CQC to overturn an unfavourable review, and threatening legal action against publications deemed to be unhelpful.
In its press release, Babylon challenges Dr Watkins to publish his research testing Babylon’s chatbot: “why not be part of an open, independent analysis of your AI testing: publish the entirety of your work, and let the totality of your data be assessed by any objective expert?”
Babylon’s challenge to Dr Watkins to openly publish his research drew open derision from some Twitter users, who pointed out that Babylon has yet to publish any randomised control studies on the safety of its chatbot in recognised peer-review journals, the accepted gold standard of clinical research.
Dr Hugh Harvey, AI co-chair of the Topol review, simply tweeted ‘Irony is not dead’.
Irony isn’t dead.@babylonhealth publicly challenges @DrMurphy11 to “publish the entirety of your work, and let the totality of your data be assessed by any objective expert” despite having no objectively assessed published data themselves…https://t.co/7mZblEpdsZ
— Hugh Harvey (@DrHughHarvey) February 24, 2020
Others noted that Babylon’s press release ends with the footnote: “Our AI tools provide information only and do not provide a medical diagnosis, nor are they a substitute for a doctor”.
Babylon also claimed it had “attempted to start a positive conversation” with Dr Watkins.
“We have invited him in to start a dialogue, to test our AI, and to meet with the senior doctors who build our products,” the statement says.
“We have corresponded with him openly and honestly when he has brought issues to our attention. We have repeatedly opened our doors, but revealingly he prefers to troll on twitter. He refused to meet us and instead posted over 6,000 misleading attacks.”
Babylon Health’s statement ends with the company inviting Dr Watkins to be part of an “open, independent analysis” of his testing.
Commenting on Babylon’s claim of “6,000 misleading attacks.”, Dr Watkins said: “Babylon Health have never previously questioned the factual accuracy of any individual tweet, nor have they challenged the factual accuracy of my presentation at the Royal Society of Medicine”.