SMART: Shaping Multilingual Access through Respeaking Technology

Summary

Our day and age is characterised by a proliferation of live multimedia and multilingual content, such as news and TV programmes. Such content is not accessible to everyone. In many countries, live subtitles in the same language are required by law to enable deaf and hard-of-hearing audiences to enjoy access to information, culture and entertainment. In the UK and other countries (e.g. Canada, Spain, Switzerland), intralingual respeaking is the most well-established technique to produce these subtitles: it relies on human-machine interaction whereby a respeaker listens to the original sound of a live programme or event and simultaneously dictates it to a speech recognition software that turns the each sentence into subtitles displayed on screen.
 

Objectives

Respeaking so far has been used to produce intralingual subtitles, i.e. in the same language. Given the current multilingual content boom, the SMART project aims to investigate whether respeaking can be used to produce interlingual subtitles, i.e. in a different language. This entails adding translation to an already challenging task which includes listening, speaking, adding oral punctuation, controlling prosody to minimise speech recognition errors, and accounting for space constraints to produce meaningful subtitles. Automatic translation alone is unable to provide live subtitles in a different language of sufficient quality. Interlingual respeaking (IRSP) represents therefore an innovative technique that builds on its monolingual predecessor to achieve this goal. This technique, however promising, has not been systematically tested yet. Therefore there is a need for empirical data to assess its viability. To this end, SMART sets out to achieve three main goals:

1. Investigate the feasibility of IRSP by analysing its challenges. Participants' performance during IRSP tasks will be studied in detail to inform optimal strategies to overcome these challenges. Two pilot projects on students have yielded promising results in this respect. We will need to explore these further in a sample of professionals;
2. Measure the quality of the live subtitles thus produced using an existing, recently-developed measurement tool (NTR model) which could represent a first step towards establishing a quality benchmark for IRSP;
3. Identify the key competencies needed by professionals already working in the language industry (e.g. interpreting, subtitling) to support timely and efficient acquisition of IRSP skills.
 

Key Impacts

IRSP has the potential to provide interlingual subtitles of virtually any live events, such as lectures, conferences, theatre shows, business meetings, live TV interviews or programmes involving speakers of different languages. IRSP could also enhance traditional subtitling methods for pre-recorded programmes, improving productivity. By doing so, IRSP would make such events and programmes accessible to everyone, irrespective of any sensory or language barriers.

Outcomes

SMART research findings will inform the design of bespoke training courses to equip language professionals with optimal skills to ensure high-quality live interlingual subtitling. We will test these materials in a Summer School during the project; after its end, we will continue to offer training that will provide language professionals with a new career path and the possibility to diversify the services they offer. A public engagement event and informative video will raise awareness of IRSP among the wider public and any other stakeholders interested in providing or benefiting from this service.

The UK pioneered the development of intralingual respeaking (today used to subtitle 85% of live TV programmes). SMART will provide the data needed for the UK to become a world leader in IRSP. IRSP is a radical evolution in respeaking methodology and a smart move towards meeting current communication needs, whilst contributing to 'barrier-free' access to different domains of public and private life for all members of society.