The challenges of ambient scribe technology in mental health

AI in mental health settings needs to be implemented thoughtfully if we are to avoid eroding patient trust, writes Jack Tabner, general manager for mental health at Accurx

Mental health teams will be the backbone of neighbourhood health, but too often they’re the last to benefit from transformative technology. 

This technological lag has tangible consequences, contributing to a staggering administrative burden that fuels burnout and pulls clinicians away from care. We’ve got to change that.

Right now, there’s excitement around ambient voice technology (AVT), and for good reason.

 It can help us leapfrog inefficiencies, reduce clinical admin, and improve the safety and consistency of care.

But in mental health, we have to proceed thoughtfully. The introduction of a new tool into the therapeutic space is an intervention in the delicate relationship between a patient and their clinician.

At Accurx, we’re obsessed with building technology that genuinely helps. 

That means we want to tackle the complex considerations that come with this technology head-on. 

That’s why we’ve channelled conversations and insights from patients and NHS mental health professionals into our new report, ‘Beyond the Hype: Navigating the Challenges of Ambient Scribe Technology in Mental Health.’

The report dives into two of the most critical challenges facing the safe and responsible use of AVT in mental health, offering a way forward that prioritises patient safety and clinical integrity. 

We don’t have all the answers, but we’re committed to asking the right questions, staying grounded in frontline reality, and helping to shape a future where mental health services get the support they deserve.

The ‘third party’ in the room

The promise of a scribe is simple: it handles the notes so the clinician can be more present with the patient  – more eye contact, better engagement, and less time buried in a computer screen.

This is a huge win. Evidence shows that clinicians spend a great deal of time on administrative tasks, which contributes to burnout and reduces the time available for direct patient care.

But here’s the rub: what happens when a patient, feels like that ‘listening’ device  is a non-human ‘third party’ in the room? 

The introduction of an AI scribe can risk eroding the trust, safety, and confidentiality between patient and clinician that are so essential to mental health care. 

This is especially true for vulnerable patients, such as those with paranoia or a history of trauma, for whom the knowledge that their conversation is being listened to and analysed by an unseen tool can be triggering.

In our report, we lay out recommendations for addressing these challenges. 

This isn’t about making a clinician’s life easier at the expense of the patient’s feeling of safety. It’s about building a tool that enhances the therapeutic bond. Our recommendations include:

• Human-in-the-loop philosophy: Clinicians should use their clinical judgment to determine if a scribe is appropriate, even with consent, and be ready to stop using it if the patient shows any discomfort.
• Privacy and control: Accurx Scribe is designed with patient safety at its core. It never uses transcripts to train the AI model, and clinicians can pause it mid-session to ensure sensitive discussions remain private, preserving a space of heightened psychological safety.
• Authentic consent: We must move beyond ‘tick-box’ consent and acknowledge the power dynamics inherent in the therapeutic relationship. A patient’s acceptance shouldn’t be mistaken for true comfort or uncoerced choice.

Sarcasm, silence, and the limits of AI

A scribe’s ability to comprehend what it hears is critical. But as we know, the language of mental health care is rarely literal. It’s a domain of nuance, subtext, and silence – the very things AI struggles with.

AI might correctly transcribe words, but completely miss the sarcasm in a patient’s voice or the meaning behind a significant pause. 

For a skilled clinician, silence can carry as much weight as words, signifying reflection, resistance, or deep emotion. But to an AI, silence is simply a lack of input.

This can lead to a new and insidious form of work for clinicians: ‘editing fatigue.’ 

When a generated note is lengthy and grammatically correct but clinically vacuous – missing essential emotional nuance or misinterpreting a sarcastic remark – the cognitive load of verifying and correcting it may be greater and more draining than writing it from scratch.

The report dives deep into these limitations and offers pragmatic solutions, for example:

• Augment, don’t automate: The clinician is always ultimately accountable for the accuracy of an AI-generated note. The scribe should be viewed strictly as a tool for generating a first draft, never a final product. Accurx Scribe is designed not to make judgments, clinical impressions, or diagnoses, ensuring the clinician remains the cornerstone of care.
• Training on AI: Clinicians need to learn how to guide an ambient scribe’s output by using prompts either during a patient encounter or when reviewing a note afterwards. Doing this effectively is an essential new skill.
• Combatting editing fatigue: Accurx Scribe is designed to minimise editing fatigue by offering features that allow for quick verbal instructions and customisable templates, ensuring the first output requires minimal editing.

Join the conversation

This report isn’t about deterring mental health providers from implementing this technology. On the contrary, it’s a call for honest, transparent dialogue. 

It’s a crucial prerequisite to the safe and responsible development and implementation of scribing tools so that mental health professionals get the powerful, thoughtfully designed tools they deserve.

We’re dedicated to listening to the concerns of clinicians and the needs of patients.

Join me and other mental health leaders on 24 September at 11am to discuss the report’s findings and share your experiences. Let’s make sure this conversation is built by the people doing the work, not just those writing the policy.