Photorealistic Human Avatars with Multi-Camera System Integration

Abstract: 
With the advent of generative AI, the future of Metaverse technologies is evolving toward breathtaking movie graphics, immersive video games, and advanced 4D content. To drive the advancement of such 4D generative AI technologies, it is essential to approach the core technology of avatar creation. Since 2019, we have embarked on a journey to achieve the perfect 4D avatar and have been working to integrate this into generative AI-based content. Our current state-of-the-art dome-type multi-camera system enables precise, high-resolution avatar capture in a multi-illuminant environment using deep learning techniques. In this keynote, I will introduce the technologies developed in our lab and discuss the future direction we should take through a comparative analysis with technologies from Meta, Google, Microsoft, and Netflix. In this presentation, I will focus on key technologies such as Multi-View Object Registration, 4D Gaussian Splatting, and 4D Light Control, with the aim of laying the groundwork for future collaboration between Yonsei University and the University of Surrey.

Speaker bio:
Sanghoon Lee received his Ph.D. in EE from the University of Texas at Austin in 2000. He worked at Korea Telecom (1991–1996) and Lucent Technologies (1999–2002). He has been serving as the Deputy Editor-in-Chief/Associate Editor of the IEEE Transactions on Multimedia (2024–/2022–) and a Member of the Senior Editorial Board of the IEEE Signal Processing Magazine (2022–). He was an Associate/Guest Editor of the IEEE Transactions on Image Processing (2010–2014, 2013), and an Associate/Senior Area Editor of the IEEE Signal Processing Letters (2014–2022). He also served as the Chair of the Asia-Pacific Signal and Information Processing Association (APSIPA) IVM Technical Committee (2018–2019), an APSIPA BoG member (2020, 2022–2024), the Editor-in-Chief of APSIPA Newsletters (2022–2023), the Chair of the IEEE P3333.1 Quality Assessment Working Group (2011–2024), and the President of the Korean Society for Simulation Surgery (2023–2024). Including service as the General Chair of the 2013 IEEE IVMSP Workshop, he has served as an organizing committee member for major conferences such as IEEE ICASSP, IEEE ICME, IEEE ICIP, and APSIPA ASC. He has also been active as a keynote speaker, invited speaker, and panelist at numerous academic conferences. He has received the Academic (2015), Contribution (2021), and Best Engineering Professor (2023) Awards from Yonsei University, the Chairman’s Award from the Presidential Council on Intellectual Property (2021), the Outstanding Area Chair Award at IEEE ICME 2020, the Best Student Paper Award at QoMEX 2018, the IEEE Transactions on Multimedia Excellent Editor Awards (2023, 2024), and the Best Demo Paper Award at ACM Multimedia 2024.