MeMAD: Methods for managing audiovisual data - Combining automatic efficiency with human accuracy
Start date
2018End date
2021Project website
ViewOverview
MeMAD is a is a H2020 research project, led by Aalto University (Finland). The project will develop novel methods and models for managing and accessing digital audiovisual information in multiple languages and for various use contexts and audiences, based on a combination of computer vision technologies, human input and machine learning approaches to derive enhanced descriptions of audio-visual content. These descriptions will benefit the Creative Industries, especially TV broadcasters and on-demand media service providers, as well as people using their services, by enabling them to access audiovisual information in novel ways.
As a partner in this project, CTS is responsible for Workpackage 5, Comparing Human and Machine-Generated Multimodal Content Description and Translation. Building on our expertise in investigating the semantic, pragmatic and discursive foundations of human audio description as an instance of multimodal translation and our experience of developing multimodal corpora, we will analyse and compare human and machine-generated descriptions of audiovisual content. The main objectives are to identify characteristic features and patterns of each method and to re-model audio description, which was originally developed as an aid for visually impaired people, into a method of describing audiovisual content for diverse audiences. The ultimate aim is to contribute to a conceptual solution for machine-assisted video description. The outcomes of this Workpackage will facilitate story-telling and the re-use of content in the broadcasting context, and improve media access for visually impaired people and other diverse groups.
Partners
- Aalto University (FI)
- University of Helsinki (FI)
- Eurecom (FR)
- University of Surrey (UK)
- YLE (FI)
- Lingsoft (FI)
- Limecraft (BE)
- Institut National de l'Audiovisuel (FR).
Principle Investigator
Professor Sabine Braun
Professor of Translation Studies; Director, Centre for Translation Studies; Co-Director, Surrey Institute for People-Centred AI
Biography
I am a Professor of Translation Studies, Director of the Centre for Translation Studies, and a Co-Director of the Surrey Institute for People-Centred Artificial Intelligence at the University of Surrey in the UK. From 2017 to 2021 I also served as Associate Dean for Research and Innovation in the Faculty of Arts and Social Sciences at the University of Surrey.
My research explores the integration and interaction of human and machine in translation and interpreting, for example to improve access to critical information, media content and vital public services such as healthcare and justice for linguistic-minority populations and other groups/people in need of communication support. My overarching interest lies in the notions of fairness, trust, transparency, and quality in relation to technology use in these contexts.
For over 10 years, I have led a programme of research that has involved cross-disciplinary collaboration with academic and non-academic partners to improve access to justice for linguistically diverse populations. Under this programme, I have investigated the use of video links in legal proceedings involving linguistic-minority participants and interpreters from a variety of theoretical and methodological perspectives. I have led several multi-national research projects in this field (AVIDICUS 1-3, 2008-16) while contributing my expertise in video interpreting to other projects in the justice sector (e.g. QUALITAS, 2012-14, Understanding Justice, 2013-16, VEJ Evaluation, 2018-20). I have advised the European Council Working Party on e-Law (e-Justice) and other justice-sector institutions in the UK and internationally on video interpreting in legal proceedings and have developed guidelines which have been reflected in European Council Recommendation 2015/C 250/01 on ‘Promoting the use of and sharing of best practices on cross-border videoconferencing’.
In other projects I have explored the use of videoconferencing and virtual reality to train users of interpreting services in how to communicate effectively through an interpreter IVY, 2011-3; EVIVA, 2014-15, SHIFT, 2015-18).
A further example of my work on accessibility is my research on audio description (video description) for visually impaired people. In the H2020 project MeMAD (2018-21) I have recently investigated the feasibility of (semi-)automating AD to improve access to media content that is not normally covered by human AD (e.g. social media content).
In 2019, the Research Centre I lead was awarded an ‘Expanding Excellence in England (E3)' grant (2019-24) by Research England to expand our research on human-machine integration in translation and interpreting. As part of this, I am currently leading and involved in a number of pilot studies aimed at better human-machine integration in different modalities of translation and interpreting.
The insights from my research have informed my teaching in interpreting and audiovisual translation on CTS’s MA programmes and the professional training courses that I have delivered (e.g. for the Metropolitan Police Service in London).
From 2018-2021 I was a member of the DIN Working Group on Interpreting Services and Technologies and co-authored the first standard on remote consecutive interpreting worldwide (DIN 8578). I am a member of the BSI Sub-committee Terminology. From 2018-2022, I was the series editor of the IATIS Yearbook (Routledge) and am currently associate series editor for interpreting of Elements in Translation and Interpreting (CUP) and a member of the Advisory Board of Interpreting (Benjamins). I was appointed to the sub-panel for Modern Languages and Linguistics for the Research Excellence Framework REF 2021.
Funder
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 780069.