MATRIC: Machine Translation and Respeaking in Interlingual Communication
Start date
2020End date
2024About the project
Summary
Interlingual communication in multilingual events (such as conferences, seminars, political events, etc.) is typically afforded via simultaneous interpretation, with highly skilled simultaneous interpreters working in the booth. However, with the advent of new technologies it has become possible to design alternative interlingual communication workflows. MATRIC is a pilot project that aims to explore to what extent communication can be ensured by a workflow involving a human respeaker who respeaks the same language into a speech recognition (SR) software to produce a clear transcript for machine translation (MT) engine.
Our experiment will look at a ‘hybrid’ scenario, where respoken output, i.e. the text recognised by the SR software, is then fed into the European Union’s machine translation engine (E-Translation) to produce translated output in 6 foreign languages. The benchmark for measuring the accuracy of communication will be the transcripts of interpretation delivered highly skilled simultaneous interpreters employed in the European institutions (in the experiment we use authentic speeches from the European Parliament for which simultaneous interpretations already exist and are publicly available in the recordings accessible via the European Parliament’s Multimedia Centre). As part of this pilot initiative, we plan to develop an evaluation framework for comparing outputs from different modes but with the same communicative objective.
Project Outcomes
The aim of the project is to see if the suggested hybrid workflow can facilitate communication in interlingual settings. An additional outcome will be an evaluation framework for comparison of interlingual and multimodal data.
The following resources were developed:
- A corpus of data from professional EN-EN respeakers aligned with data from professional interpreters.
- An evaluation interface for comparing aligned pairs of text across two different delivery modes.
- An evaluation framework for both quantitative and qualitative analysis of interlingual data delivered through different modes.
Principle Investigator
Dr Elena Davitti
Associate Professor in Translation Studies
Biography
I am an Associate Professor in Translation Studies with expertise in interpreting, both conference and dialogue. I am also Programme Leader of the MA Interpreting (Multilingual pathway) and MA Translation and Interpreting offered by the Centre for Translation Studies (CTS) where I am based. I hold a PhD in Translation and Intercultural Studies from the University of Manchester and an MA in Conference Interpreting from the University of Bologna at Forlì. Before joining Surrey in 2013, I practised as a freelance interpreter and translator and worked as interpreter trainer at different universities both in the UK and in Italy, such as the University of Leeds, University of Birmingham, University of Macerata and UNINT, Rome. I am currently working on hybrid modalities at the crossroads of traditional disciplines such as translation, interpreting , subtitling, with a particular interest in real-time speech-to-text communication across languages.
Professor Sabine Braun
Professor of Translation Studies; Director, Centre for Translation Studies; Co-Director, Surrey Institute for People-Centred AI
Biography
I am a Professor of Translation Studies, Director of the Centre for Translation Studies, and a Co-Director of the Surrey Institute for People-Centred Artificial Intelligence at the University of Surrey in the UK. From 2017 to 2021 I also served as Associate Dean for Research and Innovation in the Faculty of Arts and Social Sciences at the University of Surrey.
My research explores the integration and interaction of human and machine in translation and interpreting, for example to improve access to critical information, media content and vital public services such as healthcare and justice for linguistic-minority populations and other groups/people in need of communication support. My overarching interest lies in the notions of fairness, trust, transparency, and quality in relation to technology use in these contexts.
For over 10 years, I have led a programme of research that has involved cross-disciplinary collaboration with academic and non-academic partners to improve access to justice for linguistically diverse populations. Under this programme, I have investigated the use of video links in legal proceedings involving linguistic-minority participants and interpreters from a variety of theoretical and methodological perspectives. I have led several multi-national research projects in this field (AVIDICUS 1-3, 2008-16) while contributing my expertise in video interpreting to other projects in the justice sector (e.g. QUALITAS, 2012-14, Understanding Justice, 2013-16, VEJ Evaluation, 2018-20). I have advised the European Council Working Party on e-Law (e-Justice) and other justice-sector institutions in the UK and internationally on video interpreting in legal proceedings and have developed guidelines which have been reflected in European Council Recommendation 2015/C 250/01 on ‘Promoting the use of and sharing of best practices on cross-border videoconferencing’.
In other projects I have explored the use of videoconferencing and virtual reality to train users of interpreting services in how to communicate effectively through an interpreter IVY, 2011-3; EVIVA, 2014-15, SHIFT, 2015-18).
A further example of my work on accessibility is my research on audio description (video description) for visually impaired people. In the H2020 project MeMAD (2018-21) I have recently investigated the feasibility of (semi-)automating AD to improve access to media content that is not normally covered by human AD (e.g. social media content).
In 2019, the Research Centre I lead was awarded an ‘Expanding Excellence in England (E3)' grant (2019-24) by Research England to expand our research on human-machine integration in translation and interpreting. As part of this, I am currently leading and involved in a number of pilot studies aimed at better human-machine integration in different modalities of translation and interpreting.
The insights from my research have informed my teaching in interpreting and audiovisual translation on CTS’s MA programmes and the professional training courses that I have delivered (e.g. for the Metropolitan Police Service in London).
From 2018-2021 I was a member of the DIN Working Group on Interpreting Services and Technologies and co-authored the first standard on remote consecutive interpreting worldwide (DIN 8578). I am a member of the BSI Sub-committee Terminology. From 2018-2022, I was the series editor of the IATIS Yearbook (Routledge) and am currently associate series editor for interpreting of Elements in Translation and Interpreting (CUP) and a member of the Advisory Board of Interpreting (Benjamins). I was appointed to the sub-panel for Modern Languages and Linguistics for the Research Excellence Framework REF 2021.
Professor Constantin Orasan
Professor of Language and Translation Technologies
Biography
I am Professor of Language and Translation Technologies at the Centre of Translation Studies, University of Surrey. Before starting this role, I was Reader in Computational Linguistics at the University of Wolverhampton, UK, and the deputy head of the Research Group in Computational Linguistics at the same university. I have received my BSc in computer science at Babeș-Bolyai University, Cluj-Napoca, Romania and was awarded my PhD from the University of Wolverhampton.
I have over 20 years experience of working in the fields of (applied) Natural Language Processing (NLP), Artificial Intelligence and Machine Learning for language processing. My research interests are largely focused on facilitating information access and include translation technology, sentiment analysis, question answering, text summarisation, anaphora and coreference resolution, building, annotation and exploitation of corpora.
I recently coordinated the EXPERT project, an extremely successful Initial Training Network (ITN) funded under the People Programme of the Seventh Framework Programme (FP7) of the European Community which trained the next generation of world-class researchers in the field of data-driven translation technology. In addition to coordinating this project between nine partners across both academia and industry, I was actively involved in the training of the Early Stage Researchers (ESRs) appointed in the project and, in collaboration with these ESRs, I carried out research on translation memories and quality estimation for machine translation. I continue researching these topics.
I was also the deputy coordinator of the FIRST project, a project which developed language technologies for making texts more accessible to people with autism. In addition to managing a consortium of nine partners from academia, industry and heath care organisations, I also carried out research on text simplification and contributed to the development of a powerful editor which can be used by carers of people with autism to make texts more accessible for these people.
In the past In the past, I was the Local Course Coordinator of the Erasmus Mundus programme on Technology for Translation and Interpretation and the Erasmus Mundus International Masters in Natural Language Processing and Human Language Technology, and the scientist in charge for the University of Wolverhampton in two European projects QALL-ME and MESSAGE. I also worked as a research fellow on the CAST project.
I love programming and in my spare time I contribute to some open source projects and have my own GitHub repository.
Funder
Research England
Contact
For enquiries or potential collaboration on this topic please contact Dr Elena Davitti, the Principal Investigator of the project.
See other research projects carried out at the Centre for Translation Studies.