Multimedia Coding and Visual Quality Assessment

Multimedia coding has become a very hot research topic since 1990s, which can be clearly seen from the very active standardization activities of many standardization bodies (esp. ISO, IEC and ITU) and experts groups (esp. JPEG and MPEG). Its importance is demonstrated by applications of some multimedia coding standards such as MP3, JPEG, MPEG-2, H.264/MPEG-4 AVC in our everyday life (digital music, digital TV, digital cameras and camcorders, images and videos on Internet, etc.). Most mainstream multimedia coding standards are based on lossy compression by exploiting redundancy in human visual or aural systems. As a result, it becomes a very important task to assess the perceptual quality of the decoded multimedia data compared. The MSF group is also doing research on image and video coding, and visual quality assessment of digital images/videos and performance evaluation of digital imaging devices. This line of research has also close links to our research on multimedia security and forensics since visual quality is often a key factor in these systems.

The main research topics the MSF group is currently working on include the following:

Recovering missing information in transform-coded images and videos: Our recent research showed that missing coefficients in DCT-coded images can be recovered with surprisingly high visual quality. This has potential applications in many fields of multimedia coding, including cryptanalysis of selective multimedia encryption schemes, image and video compression, error concealment of digital images and videos, image authentication and self-restoration, steganalysis of DCT-coded images and videos, multimedia forensics and anti-forensics of digital images and videos, etc. We are currently studying possible improvements of the DCT coefficients and generalisation to other transforms.
Reconfigurable Video Coding (RVC): This is a new standard recently developed by the ISO/IEC JTC1/SC29/WG11 (better known as MPEG). Its aim is to develop an abstract framework for developing reconfigurable video codecs to support the more and more demanding development tasks of MPEG and the video coding community in general. While RVC was built for video codecs, it is actually a general-purpose framework suitable for many other data-driven processes such as cryptosystems. This group has been involved in standardization activities of RVC and its applications to multimedia security and cryptography.
Performance evaluation of digital imaging devices: While visual quality of digital images and videos is mainly used for evaluating performance of image and video codecs, it can also be used to evaluate performance of digital imaging devices. Different from traditional visual quality assessment study, when we evaluate performance of digital imaging devices, it is possible to have a more controlled environment for making images and videos. Study on this topic can find applications in source identification (a subfield of multimedia forensics).
Colorimetric characterization of displays, digital imaging devices, scanners and printers: Many devices work with colours and it is natural to ask how well they render, present or (re-)produce colours. While we can measure all possible colours a particular device can render, present or (re-)produce to answer this question, it is not convenient to handle so many colours in a huge table. Most devices involving colours are actually calibrated to work with a particular (RGB) colour space that can be linked to the device independent colour space CIE XYZ or CIE L*a*b*. However, even well calibrated devices can still deviate to the expected behavior, so evaluation and re-calibration is needed from time to time. This research topic is about characterizing the colorimetric behavior of a particular device to reflect its actual functionality. In addition to its use in device calibration, it can also find applications in device identification as the last topic.
Combination of visual quality assessment and multimedia coding: In addition to simply using visual quality assessment as a quality control tool of multimedia coding, there are several ways we can combine visual quality assessment with multimedia coding. These include quality-aware coding (including embedding of watermarks and steganography), reduced-reference visual quality assessment assisted by information embedding, perceptual multimedia encryption, and so forth.