Evaluating the role of prompt engineering in improving AI-generated audio description for factual TV/media genres
Start date
2026End date
OngoingAbout the project
This project investigates how far prompt engineering can improve AI-generated audio description (AD) for factual television content. Building on our previous evaluation that revealed both strong potential and persistent weaknesses in AI-generated AD, the project shifts attention to prompt design as a potentially fast, flexible and cost-effective way to improve output quality. The overarching aim is to determine whether adjustments in prompt phrasing, structure and stylistic guidance can increase the accuracy, relevance and narrative cohesion of AD produced by multimodal AI models capable of processing text, image, audio and video.
Aims and objectives
To achieve this, the project will re-use the existing factual TV dataset, along with the baseline human and AI-generated AD from the previous project, enabling direct comparison across versions. Prompt-engineering strategies will be developed and systematically tested, with outputs recorded in a structured log to identify effective patterns. Linguistic evaluation will examine improvements, while a focus group with AD users and, where feasible, professional describers will assess perceptual and experiential gains.
A new task will be added to the project: evaluating an existing AD platform to test the AI assistant integrated into its workflow. This assessment will examine how the platform’s built-in AI support performs on the project’s factual TV material and how this compares with outputs produced through the bespoke prompt-engineering strategies developed in the study. Findings from this platform evaluation will expand the scope of the linguistic analysis. The results will be reported alongside the comparative evaluation of baseline and newly generated AD versions.
People
Professor Sabine Braun
Professor of Translation Studies; Director, Centre for Translation Studies; Co-Director, Surrey Institute for People-Centred AI
Biography
I am a Professor of Translation Studies, Director of the Centre for Translation Studies, and a Co-Director of the Surrey Institute for People-Centred Artificial Intelligence at the University of Surrey in the UK. From 2017 to 2021 I also served as Associate Dean for Research and Innovation in the Faculty of Arts and Social Sciences at the University of Surrey.
My research explores the integration and interaction of human and machine in translation and interpreting, for example to improve access to critical information, media content and vital public services such as healthcare and justice for linguistic-minority populations and other groups/people in need of communication support. My overarching interest lies in the notions of fairness, trust, transparency, and quality in relation to technology use in these contexts.
For over 10 years, I have led a programme of research that has involved cross-disciplinary collaboration with academic and non-academic partners to improve access to justice for linguistically diverse populations. Under this programme, I have investigated the use of video links in legal proceedings involving linguistic-minority participants and interpreters from a variety of theoretical and methodological perspectives. I have led several multi-national research projects in this field (AVIDICUS 1-3, 2008-16) while contributing my expertise in video interpreting to other projects in the justice sector (e.g. QUALITAS, 2012-14, Understanding Justice, 2013-16, VEJ Evaluation, 2018-20). I have advised the European Council Working Party on e-Law (e-Justice) and other justice-sector institutions in the UK and internationally on video interpreting in legal proceedings and have developed guidelines which have been reflected in European Council Recommendation 2015/C 250/01 on ‘Promoting the use of and sharing of best practices on cross-border videoconferencing’.
In other projects I have explored the use of videoconferencing and virtual reality to train users of interpreting services in how to communicate effectively through an interpreter IVY, 2011-3; EVIVA, 2014-15, SHIFT, 2015-18).
A further example of my work on accessibility is my research on audio description (video description) for visually impaired people. In the H2020 project MeMAD (2018-21) I have recently investigated the feasibility of (semi-)automating AD to improve access to media content that is not normally covered by human AD (e.g. social media content).
In 2019, the Research Centre I lead was awarded an ‘Expanding Excellence in England (E3)' grant (2019-24) by Research England to expand our research on human-machine integration in translation and interpreting. As part of this, I am currently leading and involved in a number of pilot studies aimed at better human-machine integration in different modalities of translation and interpreting.
The insights from my research have informed my teaching in interpreting and audiovisual translation on CTS’s MA programmes and the professional training courses that I have delivered (e.g. for the Metropolitan Police Service in London).
From 2018-2021 I was a member of the DIN Working Group on Interpreting Services and Technologies and co-authored the first standard on remote consecutive interpreting worldwide (DIN 8578). I am a member of the BSI Sub-committee Terminology. From 2018-2022, I was the series editor of the IATIS Yearbook (Routledge) and am currently associate series editor for interpreting of Elements in Translation and Interpreting (CUP) and a member of the Advisory Board of Interpreting (Benjamins). I was appointed to the sub-panel for Modern Languages and Linguistics for the Research Excellence Framework REF 2021.
Professor Constantin Orasan
Professor of Language and Translation Technologies
Biography
I am Professor of Language and Translation Technologies at the Centre of Translation Studies, University of Surrey, UK, and a Fellow of the Surrey Institute for People-Centred Artificial Intelligence. Before starting this role, I was Reader (Associate Professor) in Computational Linguistics at the University of Wolverhampton, UK, and the deputy head of the Research Group in Computational Linguistics at the same university. I hold a PhD in computational linguistics and a BSc in computer science.
With over 25 years of experience in the fields of Natural Language Processing, Artificial Intelligence, and Linguistics, I have established myself as a leading researcher in the development of technologies that facilitate access to information. My PhD was in automatic summarisation, and I have led projects on question answering, text simplification, and translation technologies. Notable projects that I have led are EmpASR, an AHRC-funded project focused on training interpreters on how to benefit from the latest developments in artificial intelligence; HarnessingNLP4Court, a UKRI-funded project focused on facilitating access to legal information; the EXPERT project, an Initial Training Network (ITN) funded under the EU’s FP7 to train the next generation of world-class researchers in the field of data-driven translation technology; and the FIRST project, which developed language technologies for making texts more accessible to people with autism.
My current research is interdisciplinary, focusing on the intersection of AI, NLP, and translation studies. In recent years, I have increasingly focused on the practical application of NLP to support translators and interpreters. My recent publications explore reference-less translation evaluation, the processing of multilingual content in low-resource settings, the use of automatic speech recognition to support interpreters, and the use of large language models in text accessibility. My research is well known as a result of over 150 peer-reviewed articles in journals, books, and international conferences.
I am currently leading an EPSRC-funded project focused on making science accessible, and I am Co-Director of the ADA Leverhulme Doctoral Scholarships Network. More information about my work can be found at https://dinel.org.uk/.
Dr Yuan Zou
Lecturer in Translation Studies
Biography
I am a Lecturer in Translation Studies at the University of Surrey's Centre for Translation Studies (CTS). My background spans audiovisual translation (AVT), interpreting, and post-editing. I hold a PhD in AVT from Queen's University Belfast (QUB) and an MTI in Translation and interpreting from Jilin University.
Before joining Surrey, I was teaching Interpreting and Translation at QUB, and I engaged in freelance work as a translator and interpreter. These experiences have been instrumental in shaping my research direction and pedagogical approach.
I am currently focused on the integration of language technologies in the fields of interpreting and audiovisual translation (AVT), with a keen interest in harnessing these advancements to improve digital accessibility. I am actively investigating innovative ways in which technology can be harnessed to support and improve access for individuals with disabilities, ensuring that digital content is more inclusive and accessible to all audiences.
For enquiries or potential collaboration on this topic please contact Prof Sabine Braun, the Principal Investigator of the project.
See other research projects carried out at the Centre for Translation Studies.
Research themes
Find out more about our research at Surrey: