Research in 3D video is at the interface between computer vision and graphics. The central challenge is to exploit video capture of real-scenes to create video-realistic 3D interactive media for use in visual-effects, animation production and games. 3D video also provides the foundation for research in 3D human motion analysis in sports such as football or athletics.
Advances in multiple view video analysis have led to the introduction of techniques for 3D and free-viewpoint video reconstruction. 3D video reconstructs the shape of a dynamic scene at each video frame. Free-viewpoint video allows rendering of the scenes appearance from arbitrary viewpoints whilst aiming to preserve the video quality. Our research in free-viewpoint video for sports applications has achieved high-quality rendering in sports such as football and rugby. This allows the creation of novel virtual camera views to enhance match commentary, for example the goal keepers or referees view of a particular event or a camera view along the offside line.
V-Lab research has contributed to the high-quality reconstruction and rendering of 3D video for complex dynamic scenes captured in a controlled indoor studio or relatively uncontrolled outdoor environment such as a sports stadium. Research has also introduced techniques for stereo rendering of real scenes such as sports from a conventional monocular camera without requiring a stereo camera rig. 3D video allows highly realistic free-viewpoint action replay but cannot be modified for animation.
Our research has pioneered the used of 3D video for animation production with the introduction of surface motion graphs as a representation for combining multiple sequences recent research has introduced techniques for global alignment of entire databases of captured 3D video sequences to have a common mesh structure and known correspondence for all frames. The resulting representation is referred to as a 4D model.
Animation from 4D performance capture (temporally aligned 3D video sequences) is a central focus of our current research. Techniques have been introduced for high-level parameterisation of the captured motion allowing continuous interactive motion control. Integration of parameterised motion models into a parametric surface motion graph representation has enabled real-time interactive characters to be produced from 3D video acquisition.
V-Lab is collaborating with industry on the exploitation of 3D video technology for production of free-viewpoint video in sports, visual-effects in film and interactive characters for games. Ongoing collaboration with the BBC is investigating the use of 3D video in parallel with conventional 2D broadcast production to create interactive content.
I-Lab is conducting related research in the compression and transmission of stereo and multi-view 3D video content.
- RE@CT Immersive Production and Delivery of Interactive 3D Content (EU IST)
- SCENE Novel Scene Representations for Richer Networked Media (EU IST)