Advances in multiple view video analysis have led to the introduction of techniques for 3D and free-viewpoint video reconstruction. 3D video reconstructs the shape of a dynamic scene at each video frame. Free-viewpoint video allows rendering of the scenes appearance from arbitrary viewpoints whilst aiming to preserve the video quality.  Our research in free-viewpoint video for sports applications has achieved high-quality rendering in sports such as football and rugby. This allows the creation of novel virtual camera views to enhance match commentary, for example the goal keepers or referees view of a particular event or a camera view along the offside line.

Free-viewpoint video rendering of virtual camera view in sports TV broadcast

CVSSP V-Lab research has contributed to the high-quality reconstruction and rendering of 3D video for complex dynamic scenes captured in a controlled indoor studio or relatively uncontrolled outdoor environment such as a sports stadium. Research has also introduced techniques for stereo rendering of real scenes such as sports from a conventional monocular camera without requiring a stereo camera rig. 3D video allows highly realistic free-viewpoint action replay but cannot be modified for animation.

Our research has pioneered the used of 3D video for animation production with the introduction of surface motion graphs as a representation for combining multiple sequences recent research has introduced techniques for global alignment of entire databases of captured 3D video sequences to have a  common mesh structure and known correspondence for all frames.  The resulting representation is referred to as a 4D model.

4D model construction by global alignment of a database of 3D video sequences

Animation from 4D performance capture (temporally aligned 3D video sequences) is a central focus of our current research.  Techniques have been introduced for high-level parameterisation of the captured motion allowing continuous interactive motion control. Integration of parameterised motion models into a parametric surface motion graph representation has enabled real-time interactive characters to be produced from 3D video acquisition.

V-Lab is collaborating with industry on the exploitation of 3D video technology for production of free-viewpoint video in sports, visual-effects in film and interactive characters for games. Ongoing collaboration with the BBC is investigating the use of 3D video in parallel with conventional 2D broadcast production to create interactive content.
The CVSSP I-Lab is conducting related research in the compression and transmission of stereo and multi-view 3D video content.

Current Projects

• RE@CT Immersive Production and Delivery of Interactive 3D Content (EU IST)
• SCENE Novel Scene Representations for Richer Networked Media (EU IST)

Recent Projects

• i3Dpost Intelligent 3D content extraction and manipulation for film and games (EU IST) ( link to http://www.i3dpost.eu/)>
• VBAP Video-based animation of people (EPSRC)