News

Professor Philip Jackson (left) and Dr Davide Berghi (right) have developed a sophisticated new approach to sound localisation and detection in videos. It addresses the challenge of identifying the precise location and type of sound in stereo recordings. This is critical for applications like surveillance technology and assistive listening devices. 

To achieve this, they developed a specialised AI that acts like a fusion centre, combining information from both sound and visuals. This allows the system to understand not only what a sound is but also how far away it is. It does this by integrating pre-trained models (CLAP for audio and OWL-ViT for visuals), to incorporate rich semantic embeddings.

The research is an important step forward in enabling machines to interpret acoustic scenes in a way that closely mirrors human perception.              

Professor Jackson said: "Sound is often used to capture your attention, and good alignment between sound and vision makes an experience feel more natural; it brings the story to life. By integrating models from other areas of machine learning, we can help these networks better understand what's in the videos to hear and locate significant events more accurately." 

When submitted to the DCASE2025 Task 3 Challenge, the system performed significantly better than existing methods.

This breakthrough shows great promise for highly precise sound localisation in everyday videos.

Read the paper here

Recently, our Outreach Lead, Dr Samaneh Kouchaki led an inspiring summer school for local sixth-form students. The programme, 'Exploring Smart Tech & Future Networks,' provided 21 students from various schools with hands-on experience in cutting-edge technologies and a glimpse into future career paths.
 

Supported by the University of Surrey’s Student Success initiative, the summer school offered a unique blend of theoretical learning and practical demonstrations.

Dr. Kouchaki was supported by a team of dedicated academics and student researchers from CVSSP and CSEE, including Drs Marco Volino, Simon Hadfield, Radu Sporea, Mohsen Khalily, Shadi Danesh, and student researchers Bahar Khoram and Ahmed Bourouis. Their collective expertise ensured an inspiring learning experience for participants.

Read all about it here!

In a remarkable accomplishment for a newly graduated scientist, CVSSP’s Dr Gemma Canet Tarrés's research has been honoured at CVPR 2025, a conference recognised by Research.com as the number one machine learning/artificial intelligence conference and the top computer science event.

Dr Canet Tarrés’ research was not only lauded as a highlight paper of the show, but also featured extensively in the conference's daily newsletter. This is an outstanding achievement for Gemma, a member of our DECaDE research lab, who recently passed her viva with no corrections. This recognition provides excellent exposure for her research, which was conducted in collaboration with Adobe during her recent internship.

Dr Canet Tarres said: “I feel extremely grateful that people liked my work and found it relevant enough to be selected as a highlight at CVPR and even featured in the newsletter. I hope it can be useful, inspire others and, in one way or another, contribute to advancing the field of image generation.”

Congratulations, Gemma and team! 

👉 CVPR newsletter interview 
👉 Project Page 
👉 Arxiv 

PAI and CVSSP are immensely proud of the latest revolutionary breakthrough from our SketchX Lab, led by PAI Co-Director and CVSSP Professor Yi-Zhe Song: the groundbreaking NAG (Normalized Attention Guidance), a project led by PhD student Dar-Yen Chen. 

NAG is a technical leap that makes generative AI (especially diffusion models) far more practical and controllable for creators and developers, allowing them to quickly generate high-quality, precisely guided content.

More info here 

Supervised by PAI and CVSSP Professors Adrian Hilton and Gustavo Carneiro, PhD researcher David Butler has produced an important paper on how AI can improve osteoarthritic knee care. This is an important leap towards more personalised, anticipatory care, using AI not just to predict but to communicate future health states in a proactive way. The paper has recently been accepted to MICCAI, the leading medical image analysis conference, as being in the top 9% of the submissions, a huge accomplishment in itself.

But the researchers see the potential effect on patient outcomes as their greatest achievement. 

Pre-print of the paper that will be published in October/25: Risk Estimation of Knee Osteoarthritis Progression via Predictive Multi-task Modelling from Efficient Diffusion Model using X-ray Images

We are delighted to welcome Graham Thomas, our newly appointed Professor of Creative Technology, to CVSSP. 

“I have been fortunate enough to collaborate with Graham on numerous R&D projects over the past 20+ years. He brings a wealth of knowledge, expertise and experience in pioneering and deploying media technologies which will support the integration of Surrey’s Creative AI research across the CoSTAR National Lab.”  Professor Adrian Hilton, Director of CVSSP.

“I'm really thrilled to be joining CVSSP, after spending over 40 years working at BBC R&D, including leading research using computer vision & graphics for programme production, and overseeing work on immersive audio. I did my PhD on video motion estimation while working at the BBC back in 1990, which led to a commercially successful standards converter still in use today for conversion between 50Hz and 60Hz TV standards. I went on to work on various video coding standards, then developed some of the first technology for virtual production, including a new keying method and a camera tracking system used both for TV production and in films including Harry Potter. I have been collaborating with Adrian Hilton and the CVSSP team on various projects since 1999, with project results contributing directly to a sports graphics system that has been in regular use by the BBC and many other broadcasters since 2004. I worked with Adrian and our partners at the University of Lancaster to set up the 5-year AI4ME Prosperity Partnership on personalised media at scale, which started in September 2021. I'll be continuing to work on this now I've joined CVSSP, helping to deliver impact across the BBC and the wider media industry. I will also be working with the Surrey team in the CoSTAR programme, on technical coordination and integration with the wider programme and creative industry partners. I'll be helping with training on the UKRI AI CDT in AI for Digital Media Inclusion, and mentoring PhD and post-doctoral researchers.

 "Outside of work, I enjoy hiking and building over-complicated monitoring and control systems for my solar panels and heat pump.” Professor Graham Thomas 

Alongside her valuable work at CVSSP as a Senior Lecturer in Machine Learning for Healthcare, Dr Samaneh Kouchaki is an outreach lead for the University of Surrey: “I am engaged in several initiatives to increase awareness of our strengths while supporting student success. Recently, I delivered a 20-minute talk on AI and health at Sir William Perkins's School in Chertsey, providing insights into the intersection of artificial intelligence and healthcare applications. Additionally, I am working on organising two to three interactive sessions through the Student Success and Channel Talent outreach programme to showcase our strengths and engage with prospective students. I am looking forward to the Summer School outreach activity in July 2025, which focuses on supporting the success of a wider range of students.”

PAI and CVSSP’s Professor Mark Plumbley is the Network Lead for Noise Network Plus, a £1.8M EPSRC-funded interdisciplinary network designed to address the grand challenges for noise pollution for the next 10-15 years. 

The Network was launched in March 2025 at London’s Prince Philip House, home to the Royal Academy of Engineering. The event brought together over 100 people from academia, industry, government and the third sector. All were there with a mission: to identify the issues and challenges of noise pollution, and to plan how the Network can address them.

Professor Plumbley started the day by discussing the hard problem of noise pollution, and how the issue fits within the “Tomorrow’s Engineering Research Challenges” identified in the 2022 “TERC” Report from the Engineering and Physical Sciences Research Council (EPSRC). Delegates heard spotlight talks on areas including: noise in the built environment^[1]; new sources of noise^[2]; and effects of noise on human health^[3] and wildlife^[4]. 

The afternoon started with an introduction to the Network by Co-Leads, Professor Alan Hunter, University of Bath, and Dr. Simone Graetzer, University of Salford. This was followed by talks, including University of Surrey’s Professor Abigail Bristow’s on inclusive engineering and how it relates to noise pollution. 

In the final part of the afternoon, delegates considered how the Network can deliver lasting change to “engineer a quieter future”.

Noise Network Plus will shortly be putting out a call for Expressions of Interest in joining Working Groups. To keep up to date with announcements from Noise Network Plus, subscribe to the mailing list at https://www.jiscmail.ac.uk/NOISE

^{[1] Jack Harvie-Clark, Director Apex Acoustics} 

^{[2] Prof. Antonio J. Torija Martinez, Network Co-Lead, University of Salford}

^{[3] Dr. Benjamin Fenech, Noise and Public Health Programme Lead, UKHSA}

^{[4] Dr. Lia Gilmour, Research Manager, Bat Conservation Trust}

24 – 27 March 2025: 30 UK scientists swap places with politicians and civil servants, exploring the world of politics as part of the annual Royal Society Pairing Scheme. 

CVSSP’s Professor John Collomosse is participating in the Scheme, working directly with the Department for Science, Innovation and Technology's team for policy and capability in AI and Disinformation. This vital initiative helps ensure evidence-based policymaking. Professor Collomosse's expertise, honed through DECaDE and his work in the formation of the Content Authenticity Initiative, will be invaluable. A fantastic opportunity to amplify our research for the public good. 
 

Huge congratulations to Dr Cuong Nguyen, Professor Gustavo Carneiro (pictured), and their Monash University collaborator, Dr Toan Do, on the acceptance and selection for oral presentation at ICLR 2025 of their paper, "Probabilistic Learning to Defer: Handling Missing Expert Annotations and Controlling Workload Distribution." Only 1.8% of ICLR submitted papers are selected for an oral presentation, so this is a well-deserved great honour. 
This paper is a significant contribution from the EPSRC-funded People-Centred Mammogram Analysis (PecMan) project. It tackles the crucial challenge of developing AI models that can intelligently defer mammogram classifications to radiologists when uncertain. 
We're hugely proud of the team for their ground-breaking work. We look forward to their presentation at ICLR 2025.

In an initiative led by Professor John Collomosse, CVSSP, with PAI is contributing to a significant new research initiative aimed at advancing responsible AI within the creative sector. This project, "Performance, Participation, Provenance and Reward in Responsible AI (P3R)," is part of the Arts and Humanities Research Council’s (AHRC) Bridging Responsibilities AI Divides (BRAID) programme. 

Led by Royal Holloway, University of London (RHUL) and the CoSTAR National Lab, P3R is exploring how responsible AI tools can foster equitable opportunities for creatives and enhance environmental resilience. PAI's involvement underscores our founding tenet; to ensure AI development prioritises people-centred values and societal good.

“P3R builds upon DECaDE’s multi-year research programme in media provenance, teaming up with practice-led research in the CoStar National Lab to create new tools for fairly recognising and rewarding musicians for the re-use of their live performance.  This will help ensure an equitable creator economy in the face of advances in AI.” Professor John Collomosse

Working alongside colleagues within RHUL, Abertay University, and The National Film and Television School, Professor Collomosse’s team is contributing to the development of a demonstrator that harnesses research out of the Surrey-led DECaDE research centre to put innovative tools for content rights and monetization into the hands of musicians. This initiative aims to help them discover new audiences and address the challenges faced by grassroots music venues across the UK.