Quality of Experience
The Quality of Experience (QoE) is often used in information technology and consumer electronics domain as an indication of the overall satisfaction with the service users receive. In addition to the traditional Quality of Service (QoS), which is closely associated with the network performance, QoE also considers content quality and service quality such as cost, reliability, availability, usability, and fidelity. Hence, this is a rather subjective measure, which is immensely user dependent. As a consequence, modelling QoE objectively is quite challenging. Nevertheless, many industries, including audiovisual broadcasting industry, would be benefitted from an objective model, which can assess the QoE on the service they provide reliably, quickly and cost effectively.
There are three key contributors in the multimedia distribution chain, namely the service provider (SP), the network and the end-user. For the effective functioning of the chain, each contributor demands a sufficient level of quality. Hence comes the triple-Q concept, which was first introduced in . This concept combines the network’s Quality of Service (QoS), the customer’s Quality of Experience (QoE) and the service provider’s Quality of Business (QoB).
In the case of multimedia system deployment, the customer satisfaction does not remain only on the QoE of the delivered media, but also on the final bill the customer is asked to pay by the service provider. Apparently, there is a gap between customer satisfaction and charging. Therefore, in order to address this gap and motivate the efforts in QoE assessment, the QoB dimension is essentially attached. However, QoB has not received careful attention as a major player in the triple-Q circle.
To address the above factors, I-Lab is involved in research leading to development of QoE metrics for 3D audio-visual experience and QoB models that could be deployed in real life multimedia distribution systems. In addition to the above main stream research projects, I-Lab is also involved with fundamental research on human visual perception. The following sections briefly identify each of the research points.
3D audiovisual Quality of Experience
Ways of assessing QoE in communication environments have been discussed in recent literature. These are mainly focused on the service aspects of the communication networks, IPTV services, 2D audio-visual communications, etc. Several researchers have been working on several elements of QoE and there are also standardisation attempts to define a QoE metrics for IPTV applications. However, all QoE assessment and estimation techniques presented in the literature are for conventional 2D audiovisual applications. In contrast, the QoE studies in I-Lab, focuses on measuring the overall satisfaction on the 3D audiovisual contents. The objective is to develop a model for estimating the 3D audiovisual QoE. This model is developed by combining a set of selected QoE dimensions. The QoE dimensions and KPIs for modelling identified QoE dimensions are discussed below.
QoE for audiovisual contents is composed of a number of basic quality components. These are identified as the QoE dimensions. The spatial audio experience, 3D visual experience, and comfort are the QoE dimensions that will be considered in the MUSCADE project. These three QoE dimensions will be studied and modelled independently. Once the models for QoE dimension are available, they will be combined to obtain the overall QoE model for 3D audiovisual contents as depicted in Figure 1.
Key Performance Indicators (KPI) of the envisaged 3D Audio Visual Experience model
KPIs are used for modelling spatial audio and 3D visual experience dimensions. For modelling spatial audio experience QoE dimension, basic audio quality, front spatial fidelity and surround spatial fidelity will be used as the KPIs. Image quality and depth perception will be considered as KPIs for modelling 3D visual experience QoE dimension. The correlation between the sound and picture will be also considered for modelling the overall QoE model. This attribute includes the spatial synchronicity of audio and video, which is the correlations of source positions including azimuth, elevation and depth derived from aural and visual cues. It is known that, the user demographics, audiovisual displays available to the user and different production types may affect the user expectations, hence the QoE. These will be considered, albeit to a limited extent, defined by the availability of different 3D audio and video displays, realistic test durations and the variance in demographics of subjects that pass pre-screening. The comfort dimension will be modelled by itself from the direct subjective results acquired through a questioner.
The relationship between the QoE dimensions and KPIs is depicted in Figure 2. Spatial audio and 3D visual experience QoE dimensions and their KPIs will be investigated. The comfort dimension will also be investigated in these studies
QoE of 3D-Audio Systems
QoE investigation aims to specify the perceived quality of (spatial) audio for audiovisual contents in terms of Key Performance Indicators (KPIs), which can be related to physical parameters measurable from the contents. Four KPIs have been introduced which are believed to be useful to describe QoE objectively: Basic Audio Quality (BAQ/non-multichannel), Front and Surround Spatial Fidelity, and Audio-video Correlation. A subjective test has been conducted, in which combinations of variation in the four KPIs of audiovisual contents were presented to the listeners, who were asked to evaluate QoE. Analyses are in progress to estimate the subjective QoE score as a function of KPIs whose values can be objectively measured or calculated from the characteristics of the stimuli.
Quality of Business
We emphasize on the significant role of QoB, by arguing a “direct” impact of customer satisfaction on the payment they make for receiving a specific multimedia service. This active role of QoB aims at maintaining the revenue of SPs and guaranteeing a better quality of experience and satisfaction for users.
The current multimedia services delivery models can be illustrated by reviewing a projection of the general triple-Q model on an IPTV service delivery model. Figure 3 depicts an IPTV service delivery model introduced in  with emphasis put on the components where QoS and QoE are considered.
Observing the foregoing model, although it is believed to provide promising QoE to the user, it does not expose QoB precisely and literally. From a business point-of-view, this model focuses on the importance of user satisfaction trusting it will result in better customer loyalty, which is believed to generate more profit. From this vision, it is clear that there is no direct link between profit and customer satisfaction. Instead, it is an indirect link emerged as a result of customer loyalty. Consequently, there is a gap between customer satisfaction (QoE) and business profit (QoB). Therefore, the research on QoB at I-Lab is aimed a bridging this gap to identify comprehensive models that employ QoB as a major player.
Fundamental research on human visual perception
Psyco-physical analysis of depth perception in 3D video
Several different cues are made use by humans to perceive the depth of different objects in a scene. The depth cues can be classified mainly in to two categories, namely, oculomotor cues and visual cues. 3D video provides an additional experience of depth while watching 3D video as compared to 2D video. 3D display systems provide additional cues to its viewers that enhance the viewers’ perception of depth in a video scene. The most important one of these additional cues is the binocular disparity, which is obtained by providing two views of the same scene, from slightly different perspectives, to the each eye of a viewer. Head motion parallax is another additional cue provided by modern 3D display systems that enhances depth perception.
In this research the sensitivity of the HVS for different depth cues in 3D video is theoretically analyzed and subjectively validated. The aim of the experiments was to model how much sensitive are the humans for depth cues such as binocular disparity, retinal blur and relative size.
Effect of ambient lighting on 3D perception
Experiments are carried out to find the relationship between the background illumination and its effect upon perceptual 3D video quality and subjective depth perception in 3D video. Extensive subjective experiments are carried out for assessing the 3D video quality and depth perception with 6 video sequences at 4 different illumination levels at 4 channel bandwidths.
To summarize the results, when the ambient illumination in the usage environment for 3D video consumption increases, the video quality perception ratings of the viewers also increase. In contrast to the perceived image quality, as the ambient illumination in the environment increases, the depth perception ratings of the viewers decrease.
Psyco-physical limits of binocular suppression
It is a known property of stereoscopic video, that when one view is of high quality and the other view is of reduced resolution / low quality, the perception of the brain is dominated by the higher resolution / quality view.
In this research I-Lab is focused on obtaining the psycho-physical limits of binocular suppression at different spatial frequencies and luminance contrasts. This information will be used in a new QoE metric as well as in multiview video compression and error concealment.