Human error and ageing infrastructure is being blamed for a protracted IT pathology crash at one of Europe’s largest teaching trusts pathology IT crash.
Leeds Teaching Hospital NHS Trust has published an independent review of a major pathology outage at September last year. The crash caused the postponement of 143 non-urgent procedures, disrupted regional blood services, and knocked out part of the electronic microbiology systems for more than two months.
The review, commissioned by the trust, found the “cause of the failure was a mixture of hardware/technical failure and human error”.
It described the pathology IT system as “old, difficult to maintain and is probably in need of replacement”.
Leeds Teaching is one the busiest laboratory services in the UK, working on 17,000 samples a day and servicing the neighbouring Bradford Teaching Hospitals and 106 West Yorkshire GP practices.
The review said the outage occurred after three disks suffered hardware failures, the first on 2 August and two on 16 September. Despite each of malfunctioning disks giving “visual hardware warnings” there was “a failure to recognise and act” of that information, the report said.
Alongside the hardware problems, there was a “failure to check the completeness and integrity of back up-processes”. The two daily back-up procedures were out of sync, meaning departments that appeared later in the alphabetic listing were likely to be missed in the second back-up.
The IT pathology team also failed to “check the completeness and integrity of the second back-up” and “to recognise that the back-up times had become misaligned”.
“The situation was further compounded by the unavailability of a full back-up copy that would have allowed the system to be restored.”
Several attempts to restore the system then caused the crash between 16 and 23 September.
Leeds Teaching has been using CSC’s Telepath Laboratory Information Management System (LIMS) since the early 1980s. The report said Telepath is an IT system as its “‘end of life’ or where significant investment is needed to bring them up to modern day standards”.
In a statement provided by Digital Health News, trust chief medical officer Yvette Oade said that the “the purpose of the independent report was not to attribute blame but to identify the cause of the failure” and learn lessons.
It was not possible to estimate the cost of the failure because of its “complexity”, he said.
Oade also confirmed all the trust’s electronic pathology services were now “fully restored and working well”.
However, the trust’s corporate risk register, published in its January board papers, continues to list “unserviceable critical IT infrastructure and resilience” as a significant ongoing risk.
The review recommended the trust work closely with supplier CSC to confirm the hardware failure was “indeed coincidental and not something more sinister that could reoccur in the future”.
In a statement provided to Digital Health News, a CSC spokeswoman said the company was “fully committed to helping Leeds with the implementation of the recommendations in the report.”
During the incident, CSC “worked very closely with Leeds to help them resolve the problems”.
Other recommendations included better monitoring of staff’s actions and centrally managing the back-up processes.
Oade told Digital Health News that the trust accepted all the review recommendations and had already made several improvements. This included improving the monitoring, back-up processes, and disaster recovery within Telepath.
“Securing additional funding to invest in our IT infrastructure continues to be one of our main priorities.”
Concerns about Leeds Teaching’s IT infrastructure were flagged before the pathology crash.
In May, last year, a Care Quality Commission inspection raised concerns about the “under investment in the clinical IT systems”. While the trust’s overall rating was “good”, CQC found problems with servers, aging computers and laptops and internet access.
A multi-million-pound pathology framework tender was issued by NHS organisations in the north of England, including Leeds Teaching, in December last year.
Digital Health Intelligence maintains a database of the administrative and clinical systems in use at trusts, and uses this to calculate a clinical digital maturity index score for them. Leeds Teaching Hospitals NHS Trust (log-in required) has a score of 62 and is ranked 117 (out of 153 acute trusts).