Facial Analysis
The aim of research into Facial Analysis is to analyse and employ visual movements of facial features for the purpose of lip-reading and non-verbal communication (NVC) analysis. This starts by producing a method for efficiently and robustly tracking facial features. These include obvious parts of the face such as the eye shape, eyebrows and lip shapes (both inner and outer lips). Some of these features can be very difficult to track, for example points on the inner lip contour, due to the large amounts of visual variations that occur when a mouth opens and closes. Additionally, other features, such as the eye shape can exhibit extremely fast motions, such as blinks. To tackle the above issues, a novel technique based on selected linear predictors was proposed. Specifically a learnt person-specific but importantly, a data-driven approach to achieve accurate and real-time tracking of facial features using only intensity information.
This facial feature tracking has then been used to create automatic recognition of non-verbal communication (NVC). NVC is used in every day conversations to compliment the spoken words we use. An automatic NVC system may be useful for user product evaluations, computer game characters and learning tools. We also use unconstrained spontaneous conversations to make the data more applicable outside the lab.
Annotation of videos was conducted using public Internet worker pools (including Amazon Mechanical Turk, Samasource) to collect NVC perception data from multiple cultures. This data is publicly available as the TwoTalk corpus. This enables us to investigate cultural differences in NVC perception and to create culturally specialized NVC recognition systems.
Under controlled conditions, today's face recognition and verification algorithms can provide reasonable performance. However, in an unconstrained environment, there are various artifacts that can make recognising (or verifying) faces a difficult task. As a result faces may be captured under a variety of poses, illumination conditions, resolutions and degrees of blur, all of which provide challenges to current recognition algorithms.
Pose variation in particular is a hard problem to deal with. A widely-used approach is to attempt to warp the probe image so that it has the same pose as the gallery image. We have done this both in 2D, using an Active Appearance Model or one of its many variants, and in 3D, by fitting a 3D face model to the probe image, and then rotating it to a frontal pose to match the gallery image.
Digital Doubles: 3D Facial Performance Capture & Animation
Digital Doubles of real actors is a holy-grail of the entertainment industry from film production to interactive games. Film production of computer generated characters has achieved photo-realistic emotionally believable performance, but requires highly skilled and time-consuming manual animation. Photo-realistic animation of real people remains a challenging problem because of the importance of subtle facial detail in conveying and interpreting emotion.
CVSSP research in 3D video capture of facial performance has made possible real-time acquisition of high-definition facial shape and appearance of actor performance. Raw 3D video capture is unstructured and and can not be directly used in conventional computer graphics pipeline where artistic control is of central importance for character interaction and photo-realistic rendering.
Research, led by Prof. Adrian Hilton, supported by the Royal Society and EPSRC in collaboration with Framestore, is investigating methods for integrating 3D video capture of facial performance with visual-effects production pipelines to allow the creation of highly realistic digital doubles. This research has introduced methods for high-resolution 3D facial capture (wrinkles, pores) and techniques for temporal alignment of non-rigid mesh sequences to obtain a structured representation suitable for use in animation production.
For more details about our research in Facial Analysis, see the PDF publications below:
2D
3D
Facial Recognition
3D Facial Mapping
Tracking Facial Features
Face in 3D
(Click any image to enlarge)