Healthcare bosses should be careful when using “unstructured” Twitter feedback to measure the quality of a hospital’s care, according to a new study.
Researchers from Imperial College and Oxford University have analysed a year’s worth of tweets mentioning NHS hospitals to determine the value of social media as a means of assessing healthcare quality.
Health authorities are starting to use social networks to monitor the quality of healthcare providers, with the Department of Health’s information strategy, published in 2012, suggesting social media aggregation “[has] the potential to provide rapid indicators and early warnings of some areas of performance”.
The researchers believe their study, which was published in BMJ Quality and Safety, is the first to systematically measure tweets sent to healthcare providers to determine whether they can help healthcare providers to improve their services.
They collected nearly 200,000 tweets mentioning NHS hospitals’ official Twitter accounts over a one-year period, performing both a basic quantitative analysis of all the tweets and a qualitative analysis of 1000 that were randomly selected.
The researchers said only 11% of the tweets related to care quality, with the majority of those related to a patient’s experience with staff, facilities and access to care.
Of the tweets related to care quality, 77% were positive in tone.
Many of the tweets were related to issues other than care, such as fundraising activities, general health information and support for patients in hospital.
The study says there appears to be “no significant association” between the tone of the tweets a hospital received and traditional measures of quality, such as their risk-adjusted mortality rate and NHS inpatient survey scores.
The researchers said their findings show that healthcare authorities need to be cautious when using social media to judge the quality of services, due to the small volume of tweets related to care and the “signal to noise ratio”.
“While some tweets contain valuable information, it is unstructured, poorly labelled and hidden within a larger set of less relevant information,” says the study.
The lack of association between Twitter feedback and traditional quality metrics also shows the “frailty” of social media as a source of information on care quality, the researchers said.
However, they said the “substantial body of information” on social networks can still be useful to providers who want to highlight areas for improvement, while providing positive feedback to staff in areas where they are doing well.
“Although there are limitations to this approach, paying attention to the views of patients via social media adds to a wide collection of other information sources such as surveys, complaints and adverse incident reporting through which a hospital organisation can understand the quality and safety of the care it provides.
“In addition, if the public is aware that health services are monitoring this channel, they may become more likely to leave comments about their experiences of care.”
The researchers said further research is needed on how to collect more messages related to care, as well as how they can be used to improve the quality of services.