Long awaited and much anticipated, the final report of UCL’s independent evaluation of the Summary Care Record has just been published.
Just a look at the title – ‘The Devil’s in the Detail’ – tells you a lot of what the researchers want you to know. This is a complex issue with no simple outcomes or pat answers.
Nevertheless, the report’s discovery that there have been only modest benefits from the SCR and really no benefits from HealthSpace might lead to questions about why the two schemes should not be scrapped.
A key caveat – tucked away in a line at the beginning of the report – is that the evaluation was carried out at a time when few SCRs existed and the functionality of HealthSpace was much less than its creators hoped it would be.
This may enable those in favour of the projects to argue that the long term benefits have yet to appear – although, conversely, the argument that benefits will only come once a scheme becomes universally adopted has its own flaws.
Those in favour could also argue that emergency summary record systems have already been delivered elsewhere in the UK; so why should England not follow suit?
Although it is promising to consider the evaluation’s findings, the Department of Health sounds as if it has already made up its mind, and that the SCR programme will continue. A future for a functionally rich HealthSpace is harder to envisage.
However, if the DH decides its £1m investment in the evaluation is worth considering in detail, its final report has a wealth of information on what progress has been made so far and what it implies for the steps that will need to be taken in the future.
Use of the SCR
The three year, £1m presearch project included examination of quantative data on the creation and use of SCRs and HealthSpace, qualitative information from 237 consultations in unscheduled care, interviews with 100 stakeholders, ethnographic studies of 20 patients with long term conditions, documentary evidence such as internal reports and business plans and field notes from direct observations at more than 100 meetings and events.
A central aim of the evaluation was to look at the usability, usage, functionality and impact of the SCR and explanations for variation in its adoption and use.
The researchers found that the SCR – which was used in 22% of encounters when it was available – did appear to add value in out-of-hours consultations by helping clinicians with medication decisions.
Predictably, this was more often the case where patients were on multiple medications and were unsure of what they had been prescribed. For the advocates of the SCR this is obviously good news.
What is less positive is that the researchers did not find clear evidence that the SCR improved patient safety. The researchers found that clinicians tended to err on the side of safety – even if the SCR was not available. Benefits, the researchers, said “were subtle, hard to articulate and difficult to isolate”.
On the other hand, they saw many cases where an SCR might have added value, if it had been available, and if it had contained the data needed for a key clinical decision – in part, a function of when the research was conducted.
Similarly, the researchers point out that so far the SCR has been used mostly in primary care out-of-hours settings where it is rare for patients to encounter serious harm from medication errors.
If and when the SCR is accessed more widely in secondary care, the researchers say they cannot rule out “a significant positive impact” on patient safety. Fewer than 30 SCRs per week are currently accessed in secondary care.
On the issue of data quality, the focus of an earlier leak about the evaluation report, the researchers found SCRs do sometimes contain incomplete or inaccurate data. However, the study was not designed to identify how widespread this is.
Data quality is described by the lead author, Professor Trisha Greenhalgh, as one of the ‘wicked problems’ with the SCR that are both pervasive and seemingly insoluble.
Implementation
The report does say unequivocally that the SCR is not a plug-in technology – and it says its implementation should not be left to the IT department.
The researchers found that NHS organisations struggled with competing policy directives and a limited budget and that rapid progress was only made where there was alignment of factors.
These included top management buy-in, strong clinical engagement, an absence of powerful opponents to the scheme, and close links with key IT suppliers.
CfH’s PRINCE 2 model for implementing the programme in which explicit goals and milestones are systematically defined, pursued and signed off was described as a poor fit for this complex programme.
As a whole, IT suppliers were deemed to perceive little demand for the product from their customers, but they were prepared to participate as part of their contractual requirements and to maintain good relations with NHS Connecting for Health.
Only out of hours supplier Adastra perceived high demand for SCR functionality and saw it as a welcome part of its core business.
The researchers said they saw evidence of a more mature relationship between CfH and front line implementation staff and between CfH and GP system suppliers towards the end of their evaluation, which gave them cause for “cautious optimism”.
National versus local
An earlier evaluation of the SCR early-adopters asked whether a national system should be rejected in favour of a series of smaller, linked systems.
In ‘The Devil’s in the Detail’, the authors say the SCR is a rapidly evolving technology. Its first iteration was an upload of adverse reactions, allergies and medications, but plans are now in hand to ‘enrich’ the record and for staff in other NHS and non-NHS organisations to enter or create SCRs in various ways.
The researchers argue that ‘the many uncertainties’ implicit in these kind of extensions do not lend themselves to resolution by high-level committees. They suggest that further debate is needed at local level about what the proposals would mean for staff and patients, talking account of the particular circumstances of each local health community.
The conclusion might be that local communities must play an increasingly important part in the project but the need for a national role to provide the infrastructure is not going to go away.
HealthSpace
It is at this point that the evaluation gets most difficult because there was so little evidence for the researchers to look at on use of CfH’s online patient portal despite continuing hopes for its future.
The report says that a plan to extend HealthSpace with greatly increased functionality was initially rejected for funding, with a scaled-down version only gaining funding in 2010.
Even so, the researchers suggest that it may be time to re-visit two key assumptions; that a significant proportion of patients will have the motivation to manage their condition using this technology, and that it will reduce costs to the NHS.
The researchers found that HealthSpace users found the technology of limited value. They looked in detail at a sample of people with long term conditions who might have benefited from HealthSpace, and found that some lacked the health or IT literacy required to use it. Others had no access to computers or were using other ways of monitoring their needs.
In three pilot practices testing the HealthSpace Communicator, which allows patients to email their GP, the researchers said there were examples of patients whose care appeared to be significantly enhanced by the technology. But they also said such cases were rare.
In what might be music to a government that appeared keen in opposition on private sector offerings such as Google Health and Microsoft’s Health Vault, the report adds: “Deliberation on the future of the HealthSpace programme should take account of the availability of low-cost technologies for supporting self-management and the rapid pace of change in the market for such technologies.
“It should also reconsider the logic behind the policy-level link between ‘empowerment’ and a state-run online records service.”
Although large scale personal health record systems such as Google Health make all the headlines, the report perhaps indicates that so far benefits might more readily be seen in much smaller scale, patient led initiatives such as web forums and email lists.
Philosophically speaking
The report nears conclusion by arguing that some of the problems encountered in the SCR and HealthSpace programmes are essentially philosophical.
It argues that the programmes may have been built on the premise that sees knowledge as stable and discrete data items can be extracted from its context, placed on the SCR and transmitted to new people and contexts while retaining its meaning.
Without choosing sides, the researchers suggest others believe that knowledge is tied to particular people, organisations, experiences and practices and is difficult if not impossible to extract from its context or the people who know it.
The researchers’ final observation is that more can be achieved by continued dialogue between stakeholders – even if a hoped for consensus never materialises.