Dr Eleanor Taylor-Stilgoe

Doctoral thesis: A study of healthcare staff awareness of potential risk posed by machine translation

Eleanor Taylor-Stilgoe
Supervised by Constantin Orasan and Félix do Carmo

Published 30th April 2026

When faced with language barriers, UK healthcare staff have resorted to machine translation (MT) – predominantly Google Translate (GT) - in an attempt to balance overcoming these with fulfilling their duty of care to patients. Despite the risks potentially posed by the use of MT in such complex and sensitive situations, scant research currently exists as to healthcare staff awareness of these risks in real-life settings. This gap is particularly notable with regard to the use of MT with patient medical record documentation compared with interpersonal situations and patient-oriented documentation. While research has been conducted into the perceptions and practices of the general population around MT use in largely lower-stakes contexts, research on the extent to which these transfer to higher-stakes settings like healthcare remains lacking. Furthermore, medical abbreviations, an ubiquitous presence in healthcare environments, have previously been shown to pose an increased risk for patient harm even prior to their translation with GT. As such, abbreviations were selected as higher-risk linguistic elements to serve as a use case for the present study. Using appropriately contextualised French and Spanish data examples sourced from authoritative clinical corpora and translated using GT, these were then incorporated into semi-structured interviews with 21 healthcare staff members in diverse roles and specialties. These interview responses were then subject to in-depth qualitative analysis to ascertain the following: firstly, the extent to which healthcare staff are aware of the risks potentially posed by the use of MT with patient medical record documentation and, secondly, to identify those risks posed by the use of abbreviations when using MT in healthcare settings. 

Several significant findings were made. A major shift was observed among 93% of participants providing comparable responses about their willingness to use MT in lower- and higher-stakes settings (n=15), with 60% demonstrating a neutral attitude and 40% a positive attitude towards using MT in lower-stakes settings. When applied to patient medical record documentation, however, 53% of participants’ attitude was either fully negative or personally negative and functionally pragmatic (i.e., ‘last resort’ use). None of the participants would rely on the MT’s fitness for purpose in isolation, instead deploying a range of compensation techniques and management strategies by way of risk mitigation (e.g., referral to an available bilingual speaker or clinical senior). The abbreviations frequently went unnoticed, were actively overlooked, or were misinterpreted by participants, these issues not being specific to professional role, degree of prior MT exposure, or level of language education. Participants indicated a general reliance on and gravitation towards Romance cognate-friendlier elements of language and/or those which might be perceived as English, often at the expense of the less transparent use case abbreviations. Following discussion of the implications, participants considered the possible consequences of the latter being disregarded or misinterpreted to include the prescription of inappropriate and potentially harmful medications, treatment delays, the delivery of inappropriate treatment, and/or non-delivery of appropriate treatment, with the corresponding impacts on patient safety ranging from temporary harm requiring intervention to the death of the patient. Failure to include the full term preceding the relevant abbreviation at least once at the source reporting level was also identified as a significant contributor towards risk creation, this practice depriving the MT of vital context with which to improve the likelihood of accurate translation. Of the participants who provided feedback on the interview study process (n=18), 50% expressed a shift in their MT risk awareness, 39% had been inspired to reflect on their professional practices, and 22% expressed concern about the impact of language barriers on patient care and, in some cases, their hopes that the study would contribute towards addressing the same. These findings can inform the promotion of more informed use of MT in higher-stakes contexts, particularly those involving patient medical record documentation and/or medical abbreviations, the development of more effective risk mitigation strategies, and potentially be fed into best-practice guidelines with which to bridge the disconnect between policy, actual practice, and technological reality.

https://openresearch.surrey.ac.uk/esploro/outputs/doctoral/A-study-of-healthcare-staff-awareness/991110493802346/filesAndLinks?institution=44SUR_INST&index=0

DOI: https://doi.org/10.15126/thesis.902043

Google Translate Error Analysis for Mental Healthcare Information: Evaluating Accuracy, Comprehensibility, and Implications for Multilingual Healthcare Communication (Jaleh Delfani, Constantin Orasan, Hadeel Saadany, Ozlem Temizoz, Eleanor Taylor-Stilgoe, Diptesh Kanojia, Sabine Braun, Barbara Schouten)

Conference: Translating and the Computer - TC45 2023 · Feb 6, 2024

This study explores the use of Google Translate (GT) for translating mental healthcare (MHealth) information and evaluates its accuracy, comprehensibility, and implications for multilingual healthcare communication through analysing GT output in the MHealth domain from English to Persian, Arabic, Turkish, Romanian, and Spanish. Two datasets comprising MHealth information from the UK National Health Service website and information leaflets from The Royal College of Psychiatrists were used. Native speakers of the target languages manually assessed the GT translations, focusing on medical terminology accuracy, comprehensibility, and critical syntactic/semantic errors. GT output analysis revealed challenges in accurately translating medical terminology, particularly in Arabic, Romanian, and Persian. Fluency issues were prevalent across various languages, affecting comprehension, mainly in Arabic and Spanish. Critical errors arose in specific contexts, such as bullet-point formatting, specifically in Persian, Turkish, and Romanian. Although improvements are seen in longer-text translations, there remains a need to enhance accuracy in medical and mental health terminology and fluency, whilst also addressing formatting issues for a more seamless user experience. The findings highlight the need to use customised translation engines for Mhealth translation and the challenges when relying solely on machine-translated medical content, emphasising the crucial role of human reviewers in multilingual healthcare communication.

https://arxiv.org/abs/2402.04023

DOI: 10.48550/ARXIV.2402.04023