Sentimental audio memories
What if we captured and shared our memories and experiences through the recording of sounds rather than photographs? In this project we aim to design and test a novel audio system to answer this question.
About
We have all got used to taking photographs as visual memory triggers to remind us of the past and share with others. However, research on the sentimental properties of audio has shown that sound recordings also trigger a kind of sonic memory for events, and stimulate different kinds of conversations about the past. One of the challenges of sound recording which may have suppressed its widespread use for this purpose, is the difficulty of archiving, browsing and sharing sounds which by nature are intangible, invisible and of varying durations. In this project, we use deep learning mechanisms to automatically recognise and classify a large archive of sound recordings into categories that can be used to filter and find them again. This has resulted in the Sentimental Audio Memories system (‘SAM’) for recording and playing back these classified sounds in a tangible and attractive way. In a second phase of the work, the SAM system will now be trialled by a small number of families with children. This will test the value of the system and the usability of blending machine and human intelligence in a single playback device.
Team members
The SAM system was originally developed on the Making Sense of Sounds project, funded by a grant of £1.59m from the EPSRC Making sense of data programme (EP/N014111/1). This ran from 1.1.16 for 36 months. The system was co-designed primarily by Tijs Duel and David Frohlich with input from Christian Kroos, Philip Jackson, Yong Xu and Mark Plumbley. It was implemented by Tijs Duel, Christian Kroos, Serge Offermans & Teun Vinken. The SAM trial will now be conducted by Sarah Campbell and David Frohlich with technical support from Serge Offermans and Teun Vinken at Interactive Matter. This is funded from 1.5.20 to 12.3.21 by a platform grant to CVSSP on Audiovisual Media Research (number EP/P022529/1).
For further information on the SAM project contact David Frohlich: d.frohlich@surrey.ac.uk
Publications
Duel T, Frohlich D M, Kroos C, Xu Y, Jackson P, Plumbley M (2018) Supporting audiography: Design of a system for sentimental sound recording, classification and playback. Communications in Computer and Information Science: HCI International 2018 - Posters' Extended Abstracts 850 Scientific Publishing Services, on behalf of Springer.
Find out more
More details about the Making sense of sounds project are here