Dr Eleanor Taylor-Stilgoe

Google Translate Error Analysis for Mental Healthcare Information: Evaluating Accuracy, Comprehensibility, and Implications for Multilingual Healthcare Communication (Jaleh Delfani, Constantin Orasan, Hadeel Saadany, Ozlem Temizoz, Eleanor Taylor-Stilgoe, Diptesh Kanojia, Sabine Braun, Barbara Schouten)

Conference: Translating and the Computer - TC45 2023 · Feb 6, 2024

This study explores the use of Google Translate (GT) for translating mental healthcare (MHealth) information and evaluates its accuracy, comprehensibility, and implications for multilingual healthcare communication through analysing GT output in the MHealth domain from English to Persian, Arabic, Turkish, Romanian, and Spanish. Two datasets comprising MHealth information from the UK National Health Service website and information leaflets from The Royal College of Psychiatrists were used. Native speakers of the target languages manually assessed the GT translations, focusing on medical terminology accuracy, comprehensibility, and critical syntactic/semantic errors. GT output analysis revealed challenges in accurately translating medical terminology, particularly in Arabic, Romanian, and Persian. Fluency issues were prevalent across various languages, affecting comprehension, mainly in Arabic and Spanish. Critical errors arose in specific contexts, such as bullet-point formatting, specifically in Persian, Turkish, and Romanian. Although improvements are seen in longer-text translations, there remains a need to enhance accuracy in medical and mental health terminology and fluency, whilst also addressing formatting issues for a more seamless user experience. The findings highlight the need to use customised translation engines for Mhealth translation and the challenges when relying solely on machine-translated medical content, emphasising the crucial role of human reviewers in multilingual healthcare communication.

https://arxiv.org/abs/2402.04023

An Exploration of Risk in the Use of MT in Healthcare Settings with Abbreviations as a Use Case (Eleanor Taylor-Stilgoe, Félix do Carmo, Constantin Orasan)

Proceedings of the International Conference HiT-IT 2023 · Jul 7, 2023

When faced with language barriers, UK healthcare staff have found themselves turning to machine translation (MT) - predominantly Google Translate - to fulfil their duty of care to patients. Despite the risks potentially posed by the use of MT in such complex and sensitive situations, little research currently exists as to healthcare staff awareness of these risks in real-life settings. This gap is particularly notable concerning the use of MT with patient medical record information compared with interpersonal situations and patient-oriented documentation. While research has been conducted into the perceptions and practices of the general population concerning MT use in largely lower-stakes contexts, research on the extent to which these transfer to higher-stakes settings remains lacking. The contribution this paper aims to make is therefore twofold: to investigate the impact of MT on patient medical record documentation and to explore the extent to which healthcare staff are aware of the risks potentially posed by its use. In this paper, we selected contextualised medical abbreviation examples from authoritative French and Spanish clinical corpora to serve as a use case, abbreviations having previously been shown to pose an increased risk for patient harm even prior to their translation with Google Translate. Examples containing higher-risk MT errors were presented to healthcare staff to ascertain their perceptions and risk awareness as part of semi-structured interviews. Whilst these interviews remain ongoing, this paper presents the findings on risks identified in the use of MT with patient medical documentation, and the responses obtained thus far.

https://acl-bg.org/proceedings/2023/HiT-IT%202023/pdf/2023.hitit2023-1…