MATRIC: Machine Translation and Respeaking in Interlingual Communication
Start date
2020End date
2024About the project
Summary
Interlingual communication in multilingual events (such as conferences, seminars, political events, etc.) is typically afforded via simultaneous interpretation, with highly skilled simultaneous interpreters working in the booth. However, with the advent of new technologies it has become possible to design alternative interlingual communication workflows. MATRIC is a pilot project that aims to explore to what extent communication can be ensured by a workflow involving a human respeaker who respeaks the same language into a speech recognition (SR) software to produce a clear transcript for machine translation (MT) engine.
Our experiment will look at a ‘hybrid’ scenario, where respoken output, i.e. the text recognised by the SR software, is then fed into the European Union’s machine translation engine (E-Translation) to produce translated output in 6 foreign languages. The benchmark for measuring the accuracy of communication will be the transcripts of interpretation delivered highly skilled simultaneous interpreters employed in the European institutions (in the experiment we use authentic speeches from the European Parliament for which simultaneous interpretations already exist and are publicly available in the recordings accessible via the European Parliament’s Multimedia Centre). As part of this pilot initiative, we plan to develop an evaluation framework for comparing outputs from different modes but with the same communicative objective.
Project Outcomes
The aim of the project is to see if the suggested hybrid workflow can facilitate communication in interlingual settings. An additional outcome will be an evaluation framework for comparison of interlingual and multimodal data.
The following resources were developed:
- A corpus of data from professional EN-EN respeakers aligned with data from professional interpreters.
- An evaluation interface for comparing aligned pairs of text across two different delivery modes.
- An evaluation framework for both quantitative and qualitative analysis of interlingual data delivered through different modes.
Principle Investigator
Dr Elena Davitti
Associate Professor in Translation Studies
Biography
I am an Associate Professor in Translation Studies with expertise in interpreting, both conference and dialogue. I am also Programme Leader of the MA Interpreting (Multilingual pathway) and MA Translation and Interpreting offered by the Centre for Translation Studies (CTS) where I am based. I hold a PhD in Translation and Intercultural Studies from the University of Manchester and an MA in Conference Interpreting from the University of Bologna at Forlì. Before joining Surrey in 2013, I practised as a freelance interpreter and translator and worked as interpreter trainer at different universities both in the UK and in Italy, such as the University of Leeds, University of Birmingham, University of Macerata and UNINT, Rome. I am currently working on hybrid modalities at the crossroads of traditional disciplines such as translation, interpreting , subtitling, with a particular interest in real-time speech-to-text communication across languages.
Professor Sabine Braun
Professor of Translation Studies; Director, Centre for Translation Studies; Co-Director, Surrey Institute for People-Centred AI
Biography
I am a Professor of Translation Studies, Director of the Centre for Translation Studies, and a Co-Director of the Surrey Institute for People-Centred Artificial Intelligence at the University of Surrey in the UK. From 2017 to 2021 I also served as Associate Dean for Research and Innovation in the Faculty of Arts and Social Sciences at the University of Surrey.
My research explores the integration and interaction of human and machine in translation and interpreting, for example to improve access to critical information, media content and vital public services such as healthcare and justice for linguistic-minority populations and other groups/people in need of communication support. My overarching interest lies in the notions of fairness, trust, transparency, and quality in relation to technology use in these contexts.
For over 10 years, I have led a programme of research that has involved cross-disciplinary collaboration with academic and non-academic partners to improve access to justice for linguistically diverse populations. Under this programme, I have investigated the use of video links in legal proceedings involving linguistic-minority participants and interpreters from a variety of theoretical and methodological perspectives. I have led several multi-national research projects in this field (AVIDICUS 1-3, 2008-16) while contributing my expertise in video interpreting to other projects in the justice sector (e.g. QUALITAS, 2012-14, Understanding Justice, 2013-16, VEJ Evaluation, 2018-20). I have advised the European Council Working Party on e-Law (e-Justice) and other justice-sector institutions in the UK and internationally on video interpreting in legal proceedings and have developed guidelines which have been reflected in European Council Recommendation 2015/C 250/01 on ‘Promoting the use of and sharing of best practices on cross-border videoconferencing’.
In other projects I have explored the use of videoconferencing and virtual reality to train users of interpreting services in how to communicate effectively through an interpreter IVY, 2011-3; EVIVA, 2014-15, SHIFT, 2015-18).
A further example of my work on accessibility is my research on audio description (video description) for visually impaired people. In the H2020 project MeMAD (2018-21) I have recently investigated the feasibility of (semi-)automating AD to improve access to media content that is not normally covered by human AD (e.g. social media content).
In 2019, the Research Centre I lead was awarded an ‘Expanding Excellence in England (E3)' grant (2019-24) by Research England to expand our research on human-machine integration in translation and interpreting. As part of this, I am currently leading and involved in a number of pilot studies aimed at better human-machine integration in different modalities of translation and interpreting.
The insights from my research have informed my teaching in interpreting and audiovisual translation on CTS’s MA programmes and the professional training courses that I have delivered (e.g. for the Metropolitan Police Service in London).
From 2018-2021 I was a member of the DIN Working Group on Interpreting Services and Technologies and co-authored the first standard on remote consecutive interpreting worldwide (DIN 8578). I am a member of the BSI Sub-committee Terminology. From 2018-2022, I was the series editor of the IATIS Yearbook (Routledge) and am currently associate series editor for interpreting of Elements in Translation and Interpreting (CUP) and a member of the Advisory Board of Interpreting (Benjamins). I was appointed to the sub-panel for Modern Languages and Linguistics for the Research Excellence Framework REF 2021.
Professor Constantin Orasan
Professor of Language and Translation Technologies
Biography
I am Professor of Language and Translation Technologies at the Centre of Translation Studies, University of Surrey, UK, and a Fellow of the Surrey Institute for People-Centred Artificial Intelligence. Before starting this role, I was Reader (Associate Professor) in Computational Linguistics at the University of Wolverhampton, UK, and the deputy head of the Research Group in Computational Linguistics at the same university. I hold a PhD in computational linguistics and a BSc in computer science.
With over 25 years of experience in the fields of Natural Language Processing, Artificial Intelligence, and Linguistics, I have established myself as a leading researcher in the development of technologies that facilitate access to information. My PhD was in automatic summarisation, and I have led projects on question answering, text simplification, and translation technologies. Notable projects that I have led are EmpASR, an AHRC-funded project focused on training interpreters on how to benefit from the latest developments in artificial intelligence; HarnessingNLP4Court, a UKRI-funded project focused on facilitating access to legal information; the EXPERT project, an Initial Training Network (ITN) funded under the EU’s FP7 to train the next generation of world-class researchers in the field of data-driven translation technology; and the FIRST project, which developed language technologies for making texts more accessible to people with autism.
My current research is interdisciplinary, focusing on the intersection of AI, NLP, and translation studies. In recent years, I have increasingly focused on the practical application of NLP to support translators and interpreters. My recent publications explore reference-less translation evaluation, the processing of multilingual content in low-resource settings, the use of automatic speech recognition to support interpreters, and the use of large language models in text accessibility. My research is well known as a result of over 150 peer-reviewed articles in journals, books, and international conferences.
I am currently leading an EPSRC-funded project focused on making science accessible, and I am Co-Director of the ADA Leverhulme Doctoral Scholarships Network. More information about my work can be found at https://dinel.org.uk/.
Funder
Research England
Contact
For enquiries or potential collaboration on this topic please contact Dr Elena Davitti, the Principal Investigator of the project.
See other research projects carried out at the Centre for Translation Studies.