Journal Club: Performance of large language models on advocating the management of meningitis

Join Urs Fisch and prof Raoul Sutter from the University Hospital Basel and University of Basel in discussion with Katie MacLure (BCS Health & Care Scotland) for our live BCS BMJ HCI Journal Club hosted by Digital Health.

Synopsis

This journal club is based on a paper entitled, Performance of large language models on advocating the management of meningitis: a comparative qualitative study by Urs Fisch, Paulina Kliem, Pascale Grzonka and Raoul Sutter

The authors used a simulated clinical scenario of a patient with bacterial meningitis secondary to mastoiditis to examine the adherence of large language models (LLMs) to bacterial meningitis guidelines. This was conducted in three independent sessions to seven publicly accessible LLMs (Bard, Bing, Claude-2, GTP-3.5, GTP-4, Llama, PaLM). Responses were evaluated for adherence to good clinical practice and two international meningitis guidelines.

Join the authors to hear and discuss their findings, conclusions and plans for further research.