David Frohlich is Director of Digital World Research Centre at the University of Surrey and Professor of Interaction Design.
He joined the Centre in January 2005 to establish a new research agenda on new media innovation with social and cultural benefit. Current work includes a mixture of PhD and Research Council projects exploring digital storytelling, personal media collections and augmented paper (http://www.surrey.ac.uk/dwrc/). Prior to joining Digital World, David worked for 14 years as a senior research scientist at HP Labs, conducting design research on the future of mobile, domestic and photographic technology.
He has contributed numerous studies and patents to the field of digital photography, as indicated in two recent books entitled 'From snapshots to social media: The changing picture of domestic photography' (2011 with Sarvas) and 'Fast design, slow innovation: Audiophotography ten years on' (2016).
David has a PhD in psychology from the University of Sheffield and post-doctoral training in Conversation Analysis from the University of York. He is currently visiting researcher in the Department of Gerontology, at the Federal University of São Carlos, Brazil (UFSCAR). David has also held visiting positions at the Royal College of Art, and the Universities of York, Manchester, Sydney (UTS) and Melbourne, and is founding editor of the international journal Personal and Ubiquitous Computing.
Areas of specialism
University roles and responsibilities
- Director, Digital World Research Centre
- Impact Officer, Department of Music and Media
04 MAR 2020
The University of Surrey’s Next Generation Paper event will reveal the future of book technology
23 JAN 2018
Faculty of Arts and Social Sciences Festival of Research 2018 and ‘Technologies Impacting Life’
In the media
My background is in psychology, human computer interaction and interaction design. My research interests include tangible interfaces to computing, new media design, and user-centred innovation. In recent years at Digital World Research Centre (DWRC) I have begun to explore new design processes for local innovation and participatory design in large multidisciplinary projects such as StoryBank, Bespoke, Community Generated Media and SUS-IT. On the latter project we developed a simple method of co-design called Focusgroup+, which we are now using to develop various 'assistive media' systems for older people.
Indicators of esteem
2002-2004 Helen Hamlyn Research Centre, Royal College of Art
2000-2003 Centre for Usable Home Technology, University of York
2010-2013 Manchester Business School, University of Manchester
2013 Faculty of Design, Architecture and the Built Environment, University of Technology Sydney
2019-Present Department of Gerontology, Federal University of São Carlos, Brazil.
- Telecoms '99, Geneva. Audioprint player
- Great expectations exhibition of the best of British design and innovation in Grand Central Station, New York, 14-28th October 2002. Audiopaperclip player built into a banqueting table.
- Photokina 2000, Cologne. HP Photosmart cameras with sound
- Helen Hamlyn End-of-Year Show, Royal College of Art, 7th-17th October 2003. Audiophoto desk.
- HP European Press Conference 2004, Cos. Audiophoto desk
- Microsoft interactivity exhibit, CHI 2008, Florence. Speculative photo displays, Photoswitch, Photomesa, Photoillume
- Telling the StoryBank exhibit, British Science Association, Festival of Science 2009, University of Surrey. StoryBank system
- London Design Festival, Victoria and Albert Museum, London. 17-23rd September 2011. Bespoke: Insight journalism as a catalyst for community engagement.
- London Design Festival, Brompton Design District, 17-22nd September 2012. Interactive newsprint.
- Unbox, New Delhi, India, 7-10th February 2013. Bespoke and Interactive Newsprint.
- Innovation in action, Impact Acceleration Account event, University of Surrey, 23rd September 2014 Talking furniture box
- Lewis Elton Gallery, University of Surrey, 17th June- 2nd July 2015 Beginning audiophotography: Photographs with sound
- Paper evolutions: Exploring digital and physical paper futures, VTT Helsinki, Finland, 9th Sept 2015. Audio-photobooks
- Immersions: An exploration in photography with sound, Lewis Elton Gallery, Guildford 4-8th July 2017. Healing waters – immersive wall projection with sound
- Printing of image with related sound. Adams, Frohlich & Rix (27th July 1999). US 20020075464 (20.6.02). US 6563563 (13.5.03). EP1095312 (15.10.03)
- Storage apparatus. Murphy & Frohlich (7th September 2000). US6803856 (12.10.04).
- Digital camera with sound recording. Frohlich & Hickey (27th April 1999). WO/2000/048388 (17.8.00). EP1028583 (16.8.00). EP1151600 (7.11.01)
- Accessing a remotely stored data set and associating notes with that data set. Grosvenor & Frohlich (10th January 2002). WO/2003/058496 (17.7.03) US20040193697 (30.9.04) EP1466272 (13.10.04)
- System for capturing audio segments in a digital camera. Battles, Staudacher, Thorland & Frohlich (22nd Feb 2002) US 20030174218 (18.9.03) JP2003283904 (3.10.03)
- Photo album with provision for media playback via surface network. Adams & Frohlich (13th March 2002) US 20040008209 (15.1.04) US 6975832 (13.12.05) JP2004034692 (5.2.04)
- Associating audio and image data. Frohlich & Adams (30th April 2002) US 20040037540 (26.2.04)
- Display and manipulation of pictorial images. Pilu, Pollard & Frohlich (16th September 2002) US 20030058275 (27.3.03)
- Internet browsing system. Frohlich & Grosvenor (30th October 2002) WO/2003/038668 (8.5.03) US 20050021673 (27.1.05)
- Method and apparatus for producing video and audio-photos from static digital images. Grosvenor, Frohlich & Hall (2nd Dec 2002) US 20050008343 (13.1.05) JP2005038399 (10.2.05)
- Communication devices. Frohlich, Brown, Durrant, Lindley, Oleksik, Robson, Rumsey, Sellen & Williamson (10th January 2008) US 20090180623 (16.7.09)
- Proactive image reminding and selection method. Kiddle, Piper, Payne, Wall & Frohlich (31st March 2009) WO/2009/130110 (29.10.09)
- Method and Apparatus for Providing Dynamic Content Associated with a Document. Frohlich, Sporea & Calic. GP application 1617602.6, (18.10.16)
Digital stories are short personal films made up of a series of still images with voiceover, music and text. The technical barriers to creating such stories are falling with the use of mobile apps which make it easy to assemble story elements as audiophoto narratives on a smartphone or tablet. In this case study, we explored the potential of mobile digital storytelling in a care home context. It was used for four weeks as form of multimedia communication between formal and informal carers inside and outside the home, and a care home resident suffering from dementia. The home was located in São Carlos, Brazil as part of a larger international project called Time Matters (UK and Brazil), in which Time stands for ‘This is me’. Fifteen digital stories were made by participants in the trial, which is about one for every visit of the researchers to the care home. Stories focused mainly on the resident; capturing aspects of everyday life discussed in Visit conversations (4), documenting Social events (3) inside or outside the home, recording Therapy sessions (3) with the resident or Health reports (3) by professional carers, and forming Media albums (2) of the residents’ art or life. In general, the technology was most useful for facilitating richer conversations with the resident and other participants, and stimulating greater expressivity and creativity in the resident herself. The desire to document the resident’s current life and interests in the home for later reminiscence by their family, stands in contrast to conventional reminiscence therapy and related digital systems. These use media artefacts to stimulate reminiscence of residents’ past life outside the home.
Electronic publishing usually presents readers with book or e-book options for reading on paper or screen. In this paper, we introduce a third method of reading on paper-and-screen through the use of an augmented book (‘a-book’) with printed hotlinks than can be viewed on a nearby smartphone or other device. Two experimental versions of an augmented guide to Cornwall are shown using either optically recognised pages or embedded electronics making the book sensitive to light and touch. We refer to these as second generation (2G) and third generation (3G) paper respectively. A common architectural framework, authoring workflow and interaction model is used for both technologies, enabling the creation of two future generations of augmented books with interactive features and content. In the travel domain we use these features creatively to illustrate the printed book with local multimedia and updatable web media, to point to the printed pages from the digital content, and to record personal and web media into the book.
This paper presents an advanced process for designing “a-books”; augmented printed books with multimedia links presented on a nearby device. Although augmented paper is not new, our solution facilitates mass market use through industry standard publishing software that generates the a-book, and regular smartphones that play related digital media by optically recognising its ordinary paper pages through the phone’s built-in camera. This augmented paper strategy informs new classifications of digital content within publication design, enabling new immersive reading possibilities. Complementary affordances of print and digital, and how these are combined and harnessed by a-books in comparison to previous augmented paper concepts are first discussed. Subsequently, an explanation of the workflow for designing a-books is described. The final discussion includes implications for content creators of paper-based publishing, and future research plans.
Focus groups have traditionally been used in market and design research to obtain group reactions to product concepts. In this article we outline a simple methodological extension to this format, involving a further stage of concept re-design in smaller subgroups facilitated by a professional designer. The method was developed in the context of working with groups of older people on concepts addressing memory, identity and social communication. It is illustrated with reference to the re-design of two seeded concepts and feedback from participants themselves on the experience of taking part. © 2014 Taylor & Francis.
Paper books offer a unique physical feel, which supports the reading experience through enhanced browsing, bookmarking, freeform annotations, memory and reduced eye strain. In contrast, electronic solutions, such as tablets and e-readers, offer interactive links, updatable information, easier content sharing, and efficient collaboration. To combine the best aspects of paper and digital information for reading, we demonstrate two mechanisms for augmenting paper with light sensors that trigger digital links on a nearby smartphone. Light Tags on every page of a book are used in a first demonstration to identify which pages are open. These are replaced with an electronic Magic Bookmark in a second demonstration, avoiding the need to instrument every page.
With current digital technologies, people have large archives of digital media, such as images and audio files, but there are only limited means to include these media in creative practices of crafting and making. Nevertheless, studies have shown that crafting with digital media often makes these media more cherished and that people enjoy being creative with their digital media. This paper aims to open up the way for novel means for crafting, which include digital media in integrations with physical construction, here called 'hybrid crafting'. Notions of hybrid crafting were explored to inform the design of products or systems that may support these new crafting practices. We designed 'Materialise' - a building set that allows for the inclusion of digital images and audio files in physical constructions by using tangible building blocks that can display images or play audio files, alongside a variety of other physical components - and used this set in four hands-on creative workshops to gain insight into how people go about doing hybrid crafting; whether hybrid crafting is desirable; what the characteristics of hybrid crafting are; and how we may design to support these practices. By reflecting on the findings from these workshops, we provide concrete guidelines for the design of novel hybrid crafting products or systems that address craft context, process and result. We aim to open up the design space to designing for hybrid crafting because these new practices provide interesting new challenges and opportunities for future crafting that can lead to novel forms of creative expression. © 2013 Springer-Verlag London.
It has been 24 years since the publication of Wellner’s (1993) digital desk, demonstrating the augmentation of paper documents with projected information. Since then there have been many related developments in computing; including the world wide web, e-book readers, maturation of the augmented reality paradigm, embedded and printed electronics, and the internet of things. In this talk I draw on some of my own design explorations of augmenting paper with sound over the years, to illustrate the value of ‘audiopaper’ but also the way these explorations were rooted in the applications and technology of the day. I show that two key technologies have been important to the implementation of audiopaper over the years, and that the bigger opportunity is in connecting paper to the web. This culminates in a vision for two future generations of paper which communicate and interact with the digital devices around them
In this paper we describe a design-orientated field study in which we deploy a novel digital display device to explore the potential integration of teenage and family photo displays at home, as well as the value of situated photo display technologies for intergenerational expression. This exploration is deemed timely given the contemporary take-up of digital capture devices by teenagers and the unprecedented volume of photographic content that teens generate. Findings support integration and the display of photos on a standalone device, as well as demonstrating the interventional efficacy of the design as a resource for provoking reflection on the research subject. We also draw upon the theoretical concept of Dialogism to understand how our design mediates intergenerational relationships and interaction aesthetics relating to the notion of ‘constructive conflict’.
In this article we describe findings from a recent study in which we interviewed four British teenage girls about their photo display practices, online and offline, in family homes. We adopted a phenomenological approach to inquiry, with a particular interest in exploring how photographic representations of self and family signal self-development in emerging adulthood. Findings reveal how teens portrayed themselves differently to friends, online, and family, offline. Self-presentation to peers through photographs was managed separately from the family and largely free from parental control. The separate, online domain was used to explore alternative self-representations with real friends. Our findings appear to signal changing politics of photograph ownership and family representation between the generations.
This contribution identifies theories and practices specific to performance art for the purpose of describing a potentially fruitful area of exchange between non-representational performance and human-computer interaction (HCI). We identify three strands of current HCI research that are already working in this area of overlap, which we have termed 'performative experience design'. We then single out one of these strands, digitally augmented autobiographical performance, for further examination. Digitally augmented autobiographical performance draws on both autobiographical performance, which we see as rooted in performance and performance art, and media sharing, a field of research within HCI. Drawing on our experiences of designing a digital system for autobiographical performance, we offer a series of proposals for HCI research and applications of performative experience design. © 2013 Copyright Taylor and Francis Group, LLC.
Personal digital media such as photos, music and films play a pervasive part in contemporary life by helping us to remember the past, communicate with each other and represent our identity to others. In this chapter we explore the value of such media for supporting wellbeing in older age, drawing on concepts from literatures on art, reminiscence and music therapy. Theoretically we argue for a new category of assistive technologies involving media creation and consumption to enhance wellbeing. We propose a framework for understanding and designing such assistive media systems which highlights the interaction between media item, author and audience. This framework is then illustrated through early attempts to explore a new kind of digital story therapy for people with dementia in a residential care setting. We conclude with recommendations for the design of future ‘assistive media’ systems and experiences that might enhance not only the lives of people with dementia, but also those around them.
A new form of interaction with digital displays is described, using the pages and binding of a physical book as the interface. This leads to a form of augmented book, or a‐book, which can seamlessly trigger multimedia content on a nearby device using embedded light, pressure or touch sensors.
The use of mobile technologies appears to be in line with the strategic goals in education besides facilitating and promoting learning anywhere and anytime. However, despite the complete and advance mobile infrastructure in the developed world, the digital divide still exists in developing countries. This paper discussed the students’ behaviour and responds towards digital devices and mobile learning through interview sessions held with the school administrator and teachers. The paper defines the various perceptions of the use of mobile technology for teaching and learning by reflecting the positive opinions from the school administrator and the teachers. The different perceptions and acceptance towards technology between the aboriginal and non-aboriginal students are also reviewed in this paper.
We introduce `PaperClip' - a novel digital pen interface for semantic editing of speech recordings for radio production. We explain how we designed and developed our system, then present the results of a contextual qualitative user study of eight professional radio producers that compared editing using PaperClip to a screen-based interface and normal paper. As in many other paper-versus-screen studies, we found no overall preferences but rather advantages and disadvantages of both in different contexts. We discuss these relative benefits and make recommendations for future development.
This paper looks at the context of materialised memories - the consumption and framing of photographs. Ethnographic work in British homes unearthed diverse ways of consuming and displaying photos. We propose that these modes of framing mirror the relationships within and surrounding the household, and locate them in short-hand time frames characteristic of the social exchanges appropriate to those relationships. Through framing, people flog their collective good intentions to conduct relationships appropriately over time, without capitulating either to the risk of over-imposing nor of neglect. As a counterpart to Gell's and Strathern's analyses of art and social efficacy, our work illustrates the capacity within British family culture to materialise intention around on efficacious social object,constructing intention as a quality of persons not objects while retaining the agent-like Properties of photographs.
The user-centered design (UCD) process in HCI has recently been criticized for not delivering breakthrough innovations in technology. In this paper we consider this critique through a literature review and two case studies of innovation. Our conclusions suggest that there is nothing wrong with the attitude of usercentered design which has probably been present in all major innovations down the centuries. Rather, the practice of UCD in HCI lacks attention to business factors and long term uptake of technology in society. This compromises its impact on products and should be incorporated into the study of HCI itself.
This paper describes the development and evaluation of "weegie" an audio-photography desk featuring sounds and images inspired by the Govan area of Glasgow. It was intended to be an interactive artwork that would challenge negative preconceptions about the area. The paper describes two techniques used to consider the extent to which the piece achieved these aims. The first technique is the "personal meaning map" and taken from museum studies. The second is cultural critique drawn from the arts. Building on Gaver's  strategy of using cultural commentators for 'polyphonic' assessment it considers the extent to which perspectives drawn from the humanities and the arts can be useful in evaluating design. It argues that a more rigorous understanding of critical theory is necessary to the development of interaction design criticism.
Although numerous digital tools exist to support the capture and editing of music, less attention has been paid to supporting the creative process of music composition. In this paper we report the design of a new tool in this area, targeted specifically at collaborative composition between a composer and one or more performers. The tool is an open source ‘composer’s notebook’ app called Com-Note, which supports the creation and exchange of multimedia narratives on an Android smart phone. Requirements for the design of Com-Note were derived in a case study of the collaborative composition process, as assisted by a digital storytelling app called Com-Phone developed on another project. This involved the creation and performance of a new work for trumpet and string quartet entitled Albumleaves.
From Snapshots to Social Media describes the history and future of domestic photography as mediated by technological change. Domestic photography refers to the culture of ordinary people capturing, sharing and using photographs, and is in a particular state of flux today as photos go digital. The book argues that this digital era is the third major chapter in the 170 year history of the area; following the portrait and Kodak eras of the past. History shows that despite huge changes in photographic technology and the way it has been sold, people continue to use photographs to improve memory, support communication and reinforce identity. The future will involve a shift in the balance of these core activities and a replacement of the family album with various multimedia archives for individuals, families and communities. This raises a number of issues that should be taken into account when designing new technologies and business services in this area, including: the ownership and privacy of content, multimedia standards, home ICT infrastructure, and younger and older users of images. The book is a must for designers and engineers of imaging technology and social media who want a better understanding of the history of domestic photography in order to shape its future. It will also be of value to students and researchers in science and technology studies and visual culture, as a fascinating case study of the evolving use of photographs and photographic technology in Western society.
People capture more and more photographs leading to large personal photo collections that require much time and effort to organize. A lack of organization can have a negative effect on photo retrieval and photo sharing. In this user-centred design case study we have explored new possibilities for organizing and sharing photographs. To organize photographs the concept Living Media was created; automatic positive selection based on which photographs are viewed more often and viewed for a longer time. These photographs are apparently more interesting and therefore they will keep their appearance; less popular photographs will slowly fade to black over time. To share Living Media away from the computer the device Pearl was designed. Pearl has an integrated pico projector that projects an interactive collage of Living Media in a living room. Interaction with the collage, such as deleting unwanted photographs from the collage, gives input to the selection procedure of Living Media. Placing Pearl at a distance creates a larger projection size, suitable for sharing photographs with a group of people. Our design is evaluated in two small user studies, where we found benefits and challenges of using a combination of positive selection and pico projectors for photowork and photo sharing.
This paper contributes to the current debate about the nature of beauty and aesthetics as they apply to interactive products. Current disagreement centres around the question of whether beauty should be viewed as a continuous property of objects or as a rare emotional response to object encounters (Hassenzahl 2004, Frohlich 2004). Here we develop a new perspective of beauty as a complex psychological construct, subject to competing influences from visible object properties such as shape and colour, and invisible object associations such as perceived ease of use and brand. We introduce a new methodology for examining such constructs based on a card sorting procedure, and use it to show how 36 participants think about the beauty of 35 MP3 players. One major finding is that participants tended to evaluate the players holistically, applying similar categorisations to free sorts, beauty sorts and preference sorts. This involved a common polarisation between modern and post-modern forms as they have been found to apply to architectural styles (Wilson 1996).
In this paper we report an empirical study of the photographic portrayal of family members at home. Adopting a social psychological approach and focusing on intergenerational power dynamics, our research explores the use of domestic photo displays in family representation. Parents and their teenagers from eight families in the south of England were interviewed at home about their interpretations of both stored and displayed photos within the home. Discussions centred on particular photographs found by the participants to portray self and family in different ways. The findings show that public displays of digital photos are still curated by mothers of the households, but with more difficulty and less control than with analogue photos. In addition, teenagers both contribute and comply with this curation within the home, whilst at the same time developing additional ways of presenting their families and themselves online that are ‘unsupervised’ by the curator. We highlight the conflict of interest that is at play within teen and parent practices and consider the challenges that this presents for supporting the representation of family through the design of photo display technology.
Mobile and Web 2.0 technology have the very real potential to democratize the creation and sharing of multimedia content in developing communities, even beyond the levels currently seen in community radio and television. In this article, we report the ªndings of an exercise to test this potential in partnership with a Budikote village in southern India. We show how a system called StoryBank supported the creation of short digital stories on a text-free camera phone, and how these stories could be shared through a community repository and touch-screen display. Despite the success of a ªeld trial in which 137 stories were created and shared over a one-month period, various technical and social factors meant that the devices and content were more hierarchically managed and controlled than expected. The implications of these experiences for rural development and community-centered design are discussed.
Social media products and services have the potential to address issues of social isolation in later life, when social contact often declines. However, issues of accessibility, functionality and control appear to deter the use of existing systems by some parts of the older population. In this paper, we describe a literature review and co-design exploration to understand and address these issues. Using a methodology we call Focusgroup+, we presented new product concepts to both digitally engaged and digitally unengaged groups of older people for critique and re-design. The concepts were based on familiar devices and included a Photo Phone concept for multimedia communication, a TV Talk concept for social TV, and a Twitter Well concept for broadcast text messaging. Findings from the re-design exercise show that groups responded differently to the same concepts based on their existing skills and equipment, and took them in different design directions to accommodate common preferences for meaningful communication with relatively small groups of key contacts. This led to a diversity of both appliances and apps that better reflected the diversity of participants themselves.
Photographic practice and content has always had an intimate relationship with photographic technology. The initial invention of the camera as a device for capturing images has been followed by myriad related inventions for improving the quality, size, color, speed and appearance of images, each of which has affected the kinds of photographs taken by photographers and their aesthetic and psychological effects on audiences. Traditionally, paper was the predominant medium for displaying photographs and has itself undergone a series of parallel innovations with advances in printing technology. However, the advent of mass digital photography in the 1990s has not only seen the rise of screen-displayed photographs as an alternative to photographic prints, it has also enabled photographic content to become part of a new digital ecosystem of multi-media information and devices (Sarvas and Frohlich 2012). Whereas the technological system for doing photography in the past was a relatively stable and closed world of film exposure, processing and printing, the current system is a dynamic and open one of digital bits. The capture and representation of images in digital form allows them to be made at almost zero cost, moved between different Information and Communications Technology (ICT) devices at will, and displayed in a variety of contexts and sizes. It also allows them to be shared with other people more easily and combined with other media such as video, text, music and sound recordings.
In this paper we reflect on a body of work to develop a simpler form of digital photography. We give three examples of 'Less is More' thinking in this area which are directed and inspired by naturalistic user behaviours and reactions to design ideas. Each example happens to review the place of an old technology in the new scheme of things, and challenges a technological trend in the industry. Hence, we consider the role of sound in photography to recommend audiophotographs rather than short video clips as a new media form. We look again at the role of paper in photo sharing and recommend its support and augmentation against the trend towards screen-based viewing. Finally, we consider the role of physical souvenirs and memorabilia alongside photographs, to recommend their use as story triggers and containers, in contrast to explicit multimedia presentations. The implications for simple computing are discussed.
In this paper we describe a novel photo sharing system called 4streams. This is an ambient photo display that allows a small group of users to keep in touch through a kind of visual twitter feed of concurrent photographs from their mobile phones. The photographs of up to four users are displayed in a dynamic collage in the four quadrants of a dedicated ambient display, with photographs to each quadrant arriv- ing in real time as photographs are taken/uploaded. His- torical photos can also be browsed or played back in lock- step with each other, as a reminder of what each member of the group was doing over the same period of time. The system was trailed over seven weeks by an extended family distributed over three countries. The findings suggest that the system increases the social connection and presence be- tween children, parents and grandparents of an intergenera- tional family living apart. This was not only through ’visual status’ images of family members living in different places, but also through updates of collocated members travelling away from home, and deliberately crafted images designed to elicit responses or trigger discussions in other media. The implications of these findings for theories of photo sharing are discussed.
This article details the benefits and challenges of Insight Journalism, a community engagement and research methodology developed by the interdisciplinary Bespoke project. Based in two under-resourced urban neighbourhoods in North West England, Bespoke combined community media with participatory digital design by supporting local residents to create a series of 'old' and 'new' media outputs that were exhibited locally and used within an innovative design process. The digital designs inspired by the journalism were then built by the Bespoke team and deployed within the local area, where Insight Journalists evaluated their reception. Based on our experiences, in this article, we argue that Insight Journalism can provide a vital space for exploring salient civic and social issues, but must be understood as a process of building relationships and competencies, as well as a set of products including the mediated stories and digital designs that resulted from ongoing engagement. © 2013 Intellect Ltd Article.
It is now commonplace to capture and share images in photography as triggers for memory. In this paper we explore the possibility of using sound in the same sort of way, in a practice we call audiography. We report an initial design activity to create a system called Audio Memories comprising a ten second sound recorder, an intelligent archive for auto-classifying sound clips, and a multi-layered sound player for the social sharing of audio souvenirs around a table. The recorder and player components are essentially user experience probes that provide tangible interfaces for capturing and interacting with audio memory cues. We discuss our design decisions and process in creating these tools that harmonize user interaction and machine listening to evoke rich memories and conversations in an exploratory and open-ended way.
This study examines the ways in which multiple modern communication technologies facilitate, across time and space, the maintenance of a close interpersonal relationship between two best friends. The analysis, which focuses mainly on the openings and closings of the different types of communications, reveals a tendency for the friends to shorten openings and extend closings. However, this is possible only if the friends are fully aware of, and care about, the practical, social and emotional details of each other’s lives during periods of absence. The concomitant linguistic behaviours in their interpersonal interactions could be described as a kind of pragmatics of intimacy which cannot be achieved without the explicit and practical demonstration of that mutual care and concern.
As well as updating the manifesto for an audiophotography technology and practice, this book addresses issues in design history, the social shaping of technology and the management of innovation. In particular, it reveals the very different timescales over which design and innovation operate, and the way in which design ideas evolve across different research groups, companies and application areas. The capture of photographs with sound is a simple idea, proposed 10 years ago, that has still not become widespread. In this new edition of the seminal 2004 book on Audiophotography, the author asks “Why?” A journey through the book’s citations and related commercial products shows considerable progress in understanding the role of sound in photography, and myriad design experiments to support audiovisual storytelling as a new media form. The book is a story in itself about the “long nose of innovation”, and a lesson about the need for patience and persistence in the computer industry. To reinforce this point five of the 2004 chapters are re-published in their original form. Theses describe invariant properties of ambient musical, talking and conversational photographs, and the possibility of playback from paper as well as screen. Fast Design, Slow Innovation will be of interest to researchers and designers of new media systems and experiences, and to innovation scholars or managers looking for a ten year case study of innovation in action
Older people can often fall on the wrong side of the ‘digital divide’ in terms of accessing and enjoying new digital technology. One approach to this issue is to provide training programmes and customisation techniques for using existing technology. However, another is to re-invent technology with and for older people themselves. In this paper, I propose four recommendations for re-invention, and illustrate these with examples in the domain of digital photography.
The composition of music is a complex, creative and collaborative act. This is currently done with a range of tools including the editing of musical notation, the playing, recording and playback of musical phrases, and their verbal discussion. In this project we will bring these activities together in a single 'composer's notebook' app called Com-Note for a smart phone. This will be based on the trial and extension of an existing multimedia narrative app called Com-Phone, during the creation of a new work for trumpet and string quartet.
The exponential growth of digital photo collections, combined with the legacy of printed photographs, is leading families to experience difficulties in remembering and finding photographs. Paradoxically this creates new opportunities for the re-discovery of forgotten images. This paper reports a new study in this area, based on interviews and creative activities with ten families in the south east of England. The study found that many triggers for photo re-use were either speculative or accidental, and led people to reinterpret the meaning of photographs in the light of subsequent experience and social discussion. This suggests a need to support serendipitous browsing of photographs and a more fluid and provisional approach to the semantic tagging of personal media.
We describe the process of insight journalism, in which local amateur journalists were used to generate unique insights into the digital needs of a community. We position this as a means for communities to represent themselves to designers, both as a method of designing community technologies and as a first step towards supporting innovation at a local level. To demonstrate insight journalism, we present two case studies of community technologies that were directly inspired, informed and evaluated by journalistic content. Based on this experience, we evaluate the role that insight journalism can play in designing for communities, the particular characteristics that it lends to the design process and how it might be employed to support sustainable community innovation.
In the past few years a branch of sociology, conversation analysis, has begun to have a significant impact on the design of human*b1computer interaction (HCI). The investigation of human*b1human dialogue has emerged as a fruitful foundation for interactive system design.****This book includes eleven original chapters by leading researchers who are applying conversation analysis to HCI. The fundamentals of conversation analysis are outlined, a number of systems are described, and a critical view of their value for HCI is offered.****Computers and Conversation will be of interest to all concerned with HCI issues--from the advanced student to the professional computer scientist involved in the design and specification of interactive systems.
Background Being socially connected is related to well-being, and one way of avoiding social isolation is to deepen existing relationships. Even though existing relationships can be reinforced by regular and meaningful communication, state-of-the-art communication technologies alone do not increase the quality of social connections. Thus, there is a need for the involvement of a trained human facilitator in a network of older adults, preferably for a short period, to promote the deepening of their relationships. Objective This study aimed to evaluate the hypothesis that a human-facilitated, media-sharing social networking system can improve social connection in a small group of older people, who are more vulnerable to social isolation than most, and deepen their relationships over a period of a few weeks. Methods We conducted the design and evaluation of Media Parcels, a novel human-facilitated social networking system. Media Parcels is based on the metaphor of a facilitator collecting and delivering parcels in the physical mail. Extending the metaphor, the system supports a facilitator in designing time-based, dialogue requesting parcels from participants that bring out their memories and feelings, in collecting the parcels, wrapping them in annotations that communicate the corresponding requests, and delivering the wrapped parcel to a target person. Qualitative evaluation was carried out in 2 trials with a group of 3 people each, one with family members (children and father; aged 55, 56, and 82 years old) and the other with a group of friends (aged 72, 72, and 74 years old), over 2 weeks. In each trial, data were collected in 3 interviews (pre-, mid-, and posttrial) and via system logging. Results Collected data indicate positive social effects for deepening and developing relationships. The parcel metaphor was easily understood and the computational system was readily adopted. Preferences with regard to media production or consumption varied among participants. In the family group, children preferred receiving media parcels (because of their sentimental value) to producing them, whereas the father enjoyed both. In the friendship group, preferences varied: one enjoyed both producing and receiving, while the other two preferred one over the other. In general, participants reported a preference for the production of items of a certain type depending on the associated content. Apart from having a strong engagement with the system, participants reported feeling closer to each other than usual. Conclusions For both groups, Media Parcels was effective in promoting media sharing and social connections, resulting in deepening of existing relationships. Its design informs researchers who are attempting to promote social connection in older adults.
As we go about our everyday routines we encounter and interact with numerous physical (e.g. furniture or clothes) and digital objects (e.g. photos or e-mails). Some of these objects may be particular cherished, for example because of memories attached to them. As several studies into cherished objects have shown, we have more difficulties identifying cherished digital objects than physical ones. However, cherishing a small collection of digital objects can be beneficial; e.g. it can encourage active selection of digital objects to keep and discard. This paper presents a study that aimed to increase understanding of cherished physical and digital objects, and beyond that, of how we perceive physical and digital objects, and their advantages and disadvantages. We identified design opportunities for novel products and systems that support the creation of more cherishable digital objects by extrapolating the advantages of the physical to the digital, exploiting the reasons for cherishing digital objects, and aiming for meaningful integrations of physical and digital. © 2012 ACM.
Matt Jones and David Frohlich describe the innovative StoryBank project in a rural Indian Village.
In this paper we examine how the term ‘Audio Augmented Reality’ (AAR) is used in the literature, and how the concept is used in practice. In particular, AAR seems to refer to a variety of closely related concepts. In order to gain a deeper understanding of disparate work surrounding AAR, we present a taxonomy of these concepts and highlight both canonical examples in each category, as well as edge cases that help define the category boundaries.
In this paper we consider the current and future use of cameraphones in the context of rural South Africa, where many people do not have access to the latest models and ICT infrastructure is poor. We report a new study of cameraphone use in this setting, and the design and testing of a novel application for creating rich multimedia narratives and materials. We argue for better creative media applications on mobile platforms in this region, and greater attention to their local use. Copyright 2012 ACM.
Radio production involves editing speech-based audio using tools that represent sound using simple waveforms. Semantic speech editing systems allow users to edit audio using an automatically generated transcript, which has the potential to improve the production workflow. To investigate this, we developed a semantic audio editor based on a pilot study. Through a contextual qualitative study of five professional radio producers at the BBC, we examined the existing radio production process and evaluated our semantic editor by using it to create programmes that were later broadcast. We observed that the participants in our study wrote detailed notes about their recordings and used annotation to mark which parts they wanted to use. They collaborated closely with the presenter of their programme to structure the contents and write narrative elements. Participants reported that they often work away from the office to avoid distractions, and print transcripts so they can work away from screens. They also emphasised that listening is an important part of production, to ensure high sound quality. We found that semantic speech editing with automated speech recognition can be used to improve the radio production workflow, but that annotation, collaboration, portability and listening were not well supported by current semantic speech editing systems. In this paper, we make recommendations on how future semantic speech editing systems can better support the requirements of radio production.
This paper investigates the basis for social awareness; analysing naturalistic data to understand how people convey availability and capability to communicate in everyday interaction and how they use existing presence systems. The findings show that people in close personal relationships provide intermittent information about their activities and plans which are used to infer and negotiate future contact and communication decisions. The implications for more sophisticated cross-media communication systems are discussed.
As mobile devices are becoming more ubiquitous, it is now possible to enhance the security of the phone, as well as remote services requiring identity verification, by means of biometric traits such as fingerprint and speech. We refer to this as mobile biometry. The objective of this study is to increase the usability of mobile biometry for visually impaired users, using face as biometrics. We illustrate a scenario of a person capturing his/her own face images which are as frontal as possible. This is a challenging task for the following reasons. Firstly, a greater variation in head pose and degradation in image quality (e.g., blur, de-focus) is expected due to the motion introduced by the hand manipulation and unsteadiness. Second, for the visually impaired users, there currently exists no mechanism to provide feedback on whether a frontal face image is detected. In this paper, an audio feedback mechanism is proposed to assist the visually impaired to acquire face images of better quality. A preliminary user study suggests that the proposed audio feedback can potentially (a) shorten the acquisition time and (b) improve the success rate of face detection, especially for the non-sighted users.
The problems of multi-limb coordination and environmental control are identified as important to any theory of skilled action. It is argued that these problems are well recognised yet unrelated in the research literature and would benefit from integration. Data are presented on the acquisition of bimanual coordination showing how both problems are solved together in a manipulative task. Subjects were required to steer a screen-displayed cursor along a variety of tilted tracks using two control knobs. Analysis of the knob and cursor movements used by subjects on the task shows that with practice, movement tends to become faster, more coordinated, continuous, accurate and economical; although the exact combination of these improvements depends upon individual differences in movement strategy. The implications of these findings for theories of coordination and control are discussed. © 1988.
Conceptual and methodological problems related to Schmidt' (1975) motor schema theory are discussed. In particular, the motor schema is interpreted as representing the dynamics of the system being controlled, which may or may not be associated with a referent movement pattern. Furthermore, it is suggested that prior familiarity with a control system's dynamics is a critical but uncontrolled factor in tests of the theory, and largely accounts for their equivocal findings. These ideas are examined by two experiments in which subjects had to bimanually control the movement of a computer-displayed cursor along a track on a CRT screen. Different track orientations required different patterns of movement not entailing a single generalized motor program. Experiment 1 shows that variable track performance with a given control system, results in better transfer to novel tracks than does fixed practice. Experiment 2 demonstrates that altering the control system disrupts performance whether or not the required movements remain the same. These results indicate the need for a fundamental modification of schema theory, such that a schematic representation of effector-environment relations (effector function) is available independently of particular movement patterns used in its acquisition.
We report a user requirements study of several interfaces for the playback of sounds from photographs. The study showed that users liked listening to audiophotos when the sounds are played back from photographic prints, but as a compliment to playback on a PC. When handling prints the audio needs to be invoked manually from the print with a facility to pause the audio during playback. A handheld audioprint player was then designed to fulfill these needs, based on an embedded chip in the paper.
In order to understand the effects of interaction on User Experience with a range of MP3 players, we used a Personal Construct Theory Approach (Kelly, 1955) to elicit users’ anticipations and interpretations of their experience with technology. Sixteen participants rated four different MP3 players on a variety of user-generated constructs in the form of rating scales before and after interaction. The data revealed stability to be a dimension of constructs on a continuum between stable and volatile. The data also revealed qualitative aspects of the dynamics of User Experience as users transition between pre- to post-interaction. The implications of these findings for theory, methodology and design are discussed.
Viewpoint is a public voting device developed to allow residents in a disadvantaged community to make their voices heard through a simple, lightweight interaction. This was intended to open a new channel of communication within the community and increase community members' perception of their own efficacy. Local elected officials and community groups were able to post questions on devices located in public spaces, where residents could vote for one of two responses. Question authors were subsequently required to post a response indicating any actions to be taken. Following a two-month trial, we present our experiences and contribute guidelines for the design of public democracy tools and dimensions impacting their effectiveness, including credibility, efficacy and format.
Critical design is a powerful methodology for HCI research that contributes to personal benefit and social renewal. We propose performance studies as a way of implementing and extending critical design.
This excellent collection of readings addresses the role of technology in an ageing society, mainly through the eyes of social scientists and medical researchers using ethnographic studies of technology use. Falling broadly into the field of gerontechnology, the writings are concerned with the use of technology to improve the health and wellbeing of older people in particular. However, unlike many works in that area, they are critical of overly optimistic predictions about the value of new technologies, and more realistic about the challenges of making technology accessible, attractive and useful to people, and having it fit within everyday behaviours and broader healthcare systems. At the same time, the empirical attention to the use of technology, explodes numerous myths about the reticence of older people to engage with it. The book shows many older people to be frequent users of email and social networking systems, avid gamers, on-line shoppers and web surfers; open to exotic developments such as social robots and concerned about the inheritance of digital possessions when they die.
With the expansion of digital photographic content stored online and concurrent proliferation of capturing devices, the management and visualization of personal photo collections have become very challenging tasks. In order to gain insight into novel ways of handling and representing large personal photo collections, this paper presents results of a user experience study into novel visualizations of multiple photo streams, sourced from different individuals or capture devices. A web-based application prototype was designed and implemented offering synchronized visualization of photo streams in a single- or multi-window display layout. An experimental study was conducted with 20 users, and the results demonstrate high user demand for concurrent presentation of multiple media streams as well as recommends methods for leveraging its potential.
Direct manipulation refers to an interface design philosophy which originated in the early 1980's and now dominates the creation of modern software packages. In this chapter I update this philosophy in the light of recent studies, theories and interface innovations in this area. The main lesson of these developments is that a manual model of interaction does not always lead to direct or usable interfaces, and that conversational and mixed mode forms of interaction should be more widely considered as ways of extending the current HCI paradigm.
Embodied interaction describes how meaning in interaction is created through engagement. With this approach as a source of inspiration for three exploratory design cases this paper explores the possibilities of embodied interaction in storing, retrieving and enriching everyday memories. Following the principles of designing for embodiment, all three design cases aim at cueing memories through visual modalities like photo and video. We discuss these case studies in light of the embodied interaction and memory theory. Our findings indicate that everyday remembering may be a suitable application area for combining it with embodied interaction, because of its abstract and personal nature. © 2011 ACM.
Millions of people in developed countries routinely create and share digital content; but what about the billions of others in on the wrong side of what has been called the 'global digital divide'? This paper considers three mobile platforms to illustrate their potential in enabling rural Indian villagers to make and share digital stories. We describe our experiences in creating prototypes using mobile phones; high-end media-players; and, paper. Interaction designs are discussed along with findings from various trials within the village and elsewhere. Our approach has been to develop prototypes that can work together in an integrated fashion so that content can flow freely and in interesting ways through the village. While our work has particular relevance to those users in emerging world contexts, we see it also informing needs and practices in the developed world for user-generated content.
The paper reports a new paradigm for audiovisual information sharing in developing communities with low levels of textual and computer literacy. This was informed by ethnographic studies of a community radio station in Budikote village, India and involves the creation of audiophoto narrative stories on a mobile phone which are shared through a physical community repository (or ‘Story-bank’). The paper reports the design and evaluation of the sociotechnical system in a trial, laying the foundation for subsequent work in South Africa resulting in the Com-Me community media open source toolkit: http://digitaleconomytoolkit.org/ This work was funded by the EPSRC Bridging the Global Digital Divide initiative and led by David. . The Computer Human Interaction Conference (CHI) is the premier annual conference in the field, with full paper acceptance rates of between 10 and 15%.
The application of biometric technology has so far been top-down, driven by governments and law enforcement agencies. The low demand of this technology from the public, despite its many advantages compared to the traditional means of authentication is probably due to the lack of human factor considerations in the design process. In this work, we propose a guideline to design an interactive quality-driven feedback mechanism. The mechanism aims to improve the quality of biométrie samples during the acquisition process by putting in place objective assessment of the quality and feeding this information back to the user instantaneously, thus eliminating subjective quality judgement by the user. We illustrate the feasibility of the design methodology using face recognition as a case study. Preliminary results show that the methodology can potentially increase efficiency, effectiveness and accessibility of a biométrie system.
As design practice has become more integrated in HCI research, there are on-going discussions around the role of design in research. Design research may take different forms, among which 'Research for Design' and 'Research through Design'. While, by definition, these two differ in their focus and result- The first informs the creation of a design artefact and the second aims for a contribution to knowledge-this paper presents a case study of design research in which Research for and through Design were used iteratively to gain insight into hybrid craft-an integrated physical-digital craft form. Based on our own reflections, this paper discusses what different roles these two strategies may play depending on the research topic under study; the phase in the design process; and the level of abstraction of the research activity and knowledge gained. It thus argues that using Research for and through Design together is a powerful strategy. Copyright is held by the owner/author(s).
The paper reports the first study of the domestic soundscape and the ways in which it is used by British families. It culminates in a series of novel audio design ideas, which were also published in catalogue form and resulted in four granted patents owned by the University. The catalogue and patents are available as adjunct materials for this submission. The work was carried out in collaboration with Microsoft Research Cambridge who subsequently prototyped and tested several concepts
In this paper, we describe three purposefully provocative, digital photo display technologies designed for home settings. The three devices have been built to provoke questions around how digital photographs might be seen and interacted with in novel ways. They are also intended for speculation about the expressive resources afforded by digital technologies for displaying photos. It is hoped interactions with the devices will help researchers and designers reflect on new design possibilities. The devices are also being deployed as part of ongoing home-oriented field research.
The possibility of linking paper to digital information is enhanced by recent developments in printed electronics. In this article we report the design and evaluation of a local newspaper augmented with capacitive touch regions and an embedded Bluetooth chip working with an adjunct device. These allowed the interactive playback of associated audio and the registration of manual voting actions on the web. Design conventions inherited from paper and the web were explored by showing four different versions of an interactive newspaper to 16 community residents. The diverse responses of residents are described, outlining the potential of the approach for local journalism and recommendations for the design of interactive newsprint.
There are many practitioner books on different branches and styles of photography (e.g. landscape, studio, American, etc). Some of these even cover digital photography, as it exists today (e.g. Chambers 2001). However, there are very few research books on the meaning and practice of domestic photography, and almost none on the potential impact of digital technology on this important and widespread behaviour. Chalfen’s (1987) book is still one of the best in the former area but is badly in need of updating. A collection of readings by Lister (1995) can be counted in the latter category, but adopts a rather abstract media studies approach that fails to describe photographic practices in any detail. A new book by Lambert (2003) addresses the possible combination of digital photographs, video and voiceover in 'digital storytelling'. But this is essentially prescriptive and not connected to existing photographic practices. 'Audiophotography' combines a detailed ‘user studies’ approach to photography, with consumers’ own critiques of new media content they have generated themselves. It is therefore a new book about domestic photography and its possible transformation with digital technology. Although it focusses on the role of sound in photography, it does so in relation to a new theory of photography which is tested and refined by empirical research. Such work is timely because of current interest in new media forms and the wide variety of new photography and video products and services emerging in the consumer market. It defines a new media type, audiophotos, and how it works, and should help readers to understand the possible benefits of other innovations in the digital photography industry.
Do the gender differences found when men and women maintain personal relationships in person and on the phone also emerge when they use electronic mail? Alternately, does e-mail change these ways of interacting? The authors explore the types of relationships women and men maintain by e-mail, differences in their e-mail use locally and at a distance, and differences in the contents of messages they send. The findings are based on qualitative and quantitative data collected during a 4-year period. These data suggest that using e-mail to communicate with relatives and friends replicates preexisting gender differences. Compared to men, women find e-mail contact with friends and family more gratifying. Women are more likely than men to maintain kin relationships by e-mail. They are more likely than men to use e-mail to keep in touch with people who live far away. Women's messages sent to people far away are more filled with personal content and are more likely to be exchanged in intense burst. The fit between women's expressive styles and the features of e-mail seems to be making it especially easy for women to expand their distant social networks.
Industry analysts currently disagree about the future of domestic computing. Some predict increasing sales of home PCs while others predict the break-up of the PC into a variety of information appliances. In this paper we report a study of home PC use which illuminates this issue from the perspective of existing PC-owning families. Eleven PC-owning families from the Boston area were interviewed at home about their current PC use, their attitudes to computers and the location of technology in their homes. We found that the general purpose nature of the home PC offers something for everybody in the household, and quickly becomes an established part of family life. Indeed, it was so popular in the households we visited that it had resulted in widespread competition for PC time, and had caused parents to worry about how best to control PC and internet access and influence. These behaviours and concerns led adults and children to express quite different preferences for relocating their computing experience around the house. However in both cases the needs were for better access to multifunctional extensions of the main PC. The implications of these findings for home PC and appliance evolution are discussed.
Collect Yourselves! is a technologically mediated system that opens up the transformational possibilities of performance to small groups of non-professionals sharing their own digital photos and the stories behind them. Remarkably, their performances achieve moments of emotional and aesthetic power, but these require the performers to take risks, make themselves vulnerable, and establish connections with their audiences. We discuss the framework and methodology of our interdisciplinary approach to designing these performances (Performative Experience Design), then contextualise our discussion within recent work on the subjective experience of risk in the performance literature, from both the performer’s point of view and the audience’s. Our experiences with Collect Yourselves! argue for risk as a necessary component for rewarding and potentially transformational experiences of intermedial autobiographical performance.
At the University of Surrey (Guildford, UK), we have brought together research groups in different disciplines, with a shared interest in audio, to work on a range of collaborative research projects. In the Centre for Vision, Speech and Signal Processing (CVSSP) we focus on technologies for machine perception of audio scenes; in the Institute of Sound Recording (IoSR) we focus on research into human perception of audio quality; the Digital World Research Centre (DWRC) focusses on the design of digital technologies; while the Centre for Digital Economy (CoDE) focusses on new business models enabled by digital technology. This interdisciplinary view, across different traditional academic departments and faculties, allows us to undertake projects which would be impossible for a single research group. In this poster we will present an overview of some of these interdisciplinary projects, including projects in spatial audio, sound scene and event analysis, and creative commons audio.
A Memory Box was built to illustrate the possibility of recording and attaching stories to memorabilia kept in a box. Potential users then provided a range of ideas about what kinds of stories and objects they would keep in the box, and how they would use it. The findings confirm the value of attaching stories to souvenirs, especially in the context of gift-giving, and have implications for how this might be implemented through augmented reality interfaces.
A photo-phonographic album combining stills with field recordings from a family visit to Uzbekistan in August 2009. Many of us are used to capturing experiences and memories in photographs and video. But what of the middle ground in between? What if our cameras and smart phones captured photo sequences with sound, and saved them as simple ‘audiophotographs’ or audiovisual slideshows? And what if those sounds could be played back from ‘audiopaper’ photographs as well as screen-based devices? These possibilities are illustrated in this exhibition, through material recorded by ordinary families, amateur photographers and multimedia artists. The work is based on research by David Frohlich and colleagues at HP Labs and the University of Surrey into the practice of audiophotography and how to support it, and coincides with a new book* reviewing 10 years of developments in the field.
The StoryBank project is examining technologies and practices to allow digitally impoverished communities to take part in the user-generated content revolution. The approach involves combining mobile phones to create audio-visual stories and a touch screen display situated in a community meeting place. This paper discusses the design, evaluation and refinement of the situated display. We consider how our experiences of working with a rural Indian village community influenced design processes, principles and prototypes. The work highlights the value of community-centred design practices and prototypes in such developing-world contexts.