When carrying out a piece of research like this, perhaps the easiest questions to ask – and certainly the easiest to answer – are the “facts and figures” questions.
We certainly expect that many of the headlines over the next few days will be about the numbers in our study. The 1.2m Summary Care Records that exist so far. The fact that fewer than 20 SCRs a week were being accessed in secondary care across the country. The 17% of clinicians who didn’t access a single available summary care record.
But this is only a very small part of the story. The real findings of this evaluation are much harder to encapsulate in a table or a graph, but they make for far more interesting reading. We used the evaluation as a chance to discover where the SCR might (or might not) be used, who does (or does not) use it, and why it might (or might not) work.
To give full answers to such questions would take more than these few hundred words. In reality, it takes the several hundred pages that the report runs to. Briefly, however, we found that:
- The SCR doesn’t have a particularly strong impact on patient safety. The presence of the SCR doesn’t make clinical encounters noticeably safer, and its absence doesn’t make them any less safe. This is (at least in part) because doctors, nurses and pharmacists tended to double check data, or refer patients on for more investigation, whether SCR data are available to them or not.
- The SCR can increase clinicians’ confidence in their judgements. This seems to be because, when making a decision, clinicians like to use as much data as possible. The data on the SCR made clinicians feel more confident in their decision making, even when the information on it did not change their decision.
- Conversely, the SCR can also decrease a clinician’s confidence in their decision making. This seemed to occur where the information they were viewing is not easily understood – for example, when complex medication lists are viewed by a nurse without senior-level prescribing training. In these sorts of settings, the SCR appeared to reduce confidence and increase consultation length.
Although we counted many things, answers of this sort of complexity require more than just counting. In the process of the evaluation, members of the team spent many hours watching doctors, nurses and other health professionals at work, interviewed around 140 patients and scrutinised emails, reports and business plans.
Starting to make sense of all this data is challenging but, as in the numbers game, the first thing we do is look for patterns in the data. However, we also spend a lot of time looking at the outliers, the exceptions and the unexpected, to see where our generalisations might not be that general after all.
For example, in the BMJ paper summarising this evaluation, we present a graph and some descriptive statistics that clearly demonstrate there is a large amount of variance in how often individual clinicians access the SCR, with the top fifth of the clinicians accessing more than 40% of the SCRs available to them and the bottom fifth accessing the SCR rarely or not at all.
We could have been satisfied with reporting the numbers, but the data prompted a more complex question – one that could not be satisfactorily addressed using numbers alone: why is there such variance in the numbers?
Closer observation of the encounters revealed something that we found surprising: the type of clinician was the most important factor we could identify – more important than the type of complaint certainly. Our observations showed that highly competent clinicians with high numbers of contact hours used the SCR the most, and less competent clinicians, and those who work few hours or out-of-hours, do not access SCRs even when these are easily available. Crudely put, the more conscientious the clinician, the more likely they were to use every source of data they had to hand.
The headlines you’re most likely to read today are that the SCR may save a (very) few lives but that in the main there is likely to be little impact on safety or quality of care. To understand why IT projects like this so often don’t live up to expectations, though, we have to look beyond the numbers.
About the author: Emma Byrne is a researcher with a PhD in computer science who worked on the independent evaluations of the Summary Care Record at UCL under Professor Trisha Greenhalgh.
More about the report: Read more about the report in Fiona Barr’s news story and opinion and analysis feature.