Research collaboration between Surrey and Disney gives directors control over performances in post-production
FaceDirector, a system developed by researchers at the University of Surrey and Disney Research, has made it possible for filmmakers to artificially generate desired performances for film in post-production.
In the film industry, it’s quite common for a director to re-shoot a crucial scene numerous times, spending both time and money to ensure that timing, emotion and facial expressions are near perfection.
However, a new system developed by researchers, including Surrey PhD student Charles Malleson, at the University of Surrey’s Centre for Vision, Speech and Signal Processing (CVSSP) and Disney Research has eliminated the need for excessive reshooting, by allowing multiple video takes to be blended in order to achieve the perfect performance.
FaceDirector enables a director to seamlessly blend between multiple facial performances of an actor, in which facial expressions or emotional states may differ, in order to fine-tune performances. This gives a director the power to exert control over an actor’s performance, even after the shoot, with just a few takes.
The software achieves this by allowing directors to choose desired facial expressions and timings from two separate videos, blending them together by using a mixture of facial landmarks, optical flow and compositing. Moreover, the system works with the 2D video input belonging to standard cameras, without the need for additional hardware or 3D face reconstruction.
To test the system, actors performed several lines of dialogue multiple times, each time conveying a different emotion. The line readings were captured using standard compact cameras and the researchers were able to synchronize the videos automatically, and in real-time, on a standard desktop computer. The users found that they could successfully create fresh versions of the performances by interactively blending the video takes.
“FaceDirector is the result of a highly successful collaboration between Disney Research Zurich and the Centre for Vision, Speech and Signal Processing (CVSSP) at Surrey, resulting in a new technology for more flexible actor performance production in movies,” commented Surrey's Professor Adrian Hilton. “CVSSP has led the UK in collaborative computer-vision research with the creative industries for over a decade and it is particularly rewarding to see the results of cutting-edge research being exploited by industry through joint collaboration.”
The researchers presented their findings at ICCV 2015, the International Conference on Computer Vision, between 11 and 18 December, in Santiago, Chile.