QoE Beyond the MOS: An In-Depth Look at QoE via Better Metrics and their Relation to MOS

Tobias Hoßfeld (Corresponding Author), Poul Heegaard, Martín Varela, Sebastian Möller

Research output: Contribution to journalArticleScientificpeer-review

Abstract

While Quality of Experience (QoE) has advanced very significantly as a field in recent years, the methods used for analyzing it have not always kept pace. When QoE is studied, measured or estimated, practically all the literature deals with the so-called Mean Opinion Score (MOS). The MOS provides a simple scalar value for QoE, but it has several limitations, some of which are made clear in its name: for many applications, just having a mean value is not sufficient. For service and content providers in particular, it is more interesting to have an idea of how the scores are distributed, so as to ensure that a certain portion of the user population is experiencing satisfactory levels of quality, thus reducing churn. In this article we put forward the limitations of MOS, present other statistical tools that provide a much more comprehensive view of how quality is perceived by the users, and illustrate it all by analyzing the results of several subjective studies with these tools.
Original languageEnglish
Number of pages23
JournalQuality and User Experience
Volume1
Issue number1
DOIs
Publication statusPublished - 2016
MoE publication typeA1 Journal article-refereed

Keywords

  • quality of experience
  • mean opinion score
  • metrics
  • statistics
  • quantiles
  • acceptability
  • rating distribution

Cite this

Hoßfeld, Tobias ; Heegaard, Poul ; Varela, Martín ; Möller, Sebastian. / QoE Beyond the MOS: An In-Depth Look at QoE via Better Metrics and their Relation to MOS. In: Quality and User Experience. 2016 ; Vol. 1, No. 1.
@article{3a33890675644c64a4c4211aafd11f0f,
title = "QoE Beyond the MOS: An In-Depth Look at QoE via Better Metrics and their Relation to MOS",
abstract = "While Quality of Experience (QoE) has advanced very significantly as a field in recent years, the methods used for analyzing it have not always kept pace. When QoE is studied, measured or estimated, practically all the literature deals with the so-called Mean Opinion Score (MOS). The MOS provides a simple scalar value for QoE, but it has several limitations, some of which are made clear in its name: for many applications, just having a mean value is not sufficient. For service and content providers in particular, it is more interesting to have an idea of how the scores are distributed, so as to ensure that a certain portion of the user population is experiencing satisfactory levels of quality, thus reducing churn. In this article we put forward the limitations of MOS, present other statistical tools that provide a much more comprehensive view of how quality is perceived by the users, and illustrate it all by analyzing the results of several subjective studies with these tools.",
keywords = "quality of experience, mean opinion score, metrics, statistics, quantiles, acceptability, rating distribution",
author = "Tobias Ho{\ss}feld and Poul Heegaard and Mart{\'i}n Varela and Sebastian M{\"o}ller",
year = "2016",
doi = "10.1007/s41233-016-0002-1",
language = "English",
volume = "1",
journal = "Quality and User Experience",
issn = "2366-0139",
publisher = "Springer",
number = "1",

}

QoE Beyond the MOS: An In-Depth Look at QoE via Better Metrics and their Relation to MOS. / Hoßfeld, Tobias (Corresponding Author); Heegaard, Poul; Varela, Martín; Möller, Sebastian.

In: Quality and User Experience, Vol. 1, No. 1, 2016.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - QoE Beyond the MOS: An In-Depth Look at QoE via Better Metrics and their Relation to MOS

AU - Hoßfeld, Tobias

AU - Heegaard, Poul

AU - Varela, Martín

AU - Möller, Sebastian

PY - 2016

Y1 - 2016

N2 - While Quality of Experience (QoE) has advanced very significantly as a field in recent years, the methods used for analyzing it have not always kept pace. When QoE is studied, measured or estimated, practically all the literature deals with the so-called Mean Opinion Score (MOS). The MOS provides a simple scalar value for QoE, but it has several limitations, some of which are made clear in its name: for many applications, just having a mean value is not sufficient. For service and content providers in particular, it is more interesting to have an idea of how the scores are distributed, so as to ensure that a certain portion of the user population is experiencing satisfactory levels of quality, thus reducing churn. In this article we put forward the limitations of MOS, present other statistical tools that provide a much more comprehensive view of how quality is perceived by the users, and illustrate it all by analyzing the results of several subjective studies with these tools.

AB - While Quality of Experience (QoE) has advanced very significantly as a field in recent years, the methods used for analyzing it have not always kept pace. When QoE is studied, measured or estimated, practically all the literature deals with the so-called Mean Opinion Score (MOS). The MOS provides a simple scalar value for QoE, but it has several limitations, some of which are made clear in its name: for many applications, just having a mean value is not sufficient. For service and content providers in particular, it is more interesting to have an idea of how the scores are distributed, so as to ensure that a certain portion of the user population is experiencing satisfactory levels of quality, thus reducing churn. In this article we put forward the limitations of MOS, present other statistical tools that provide a much more comprehensive view of how quality is perceived by the users, and illustrate it all by analyzing the results of several subjective studies with these tools.

KW - quality of experience

KW - mean opinion score

KW - metrics

KW - statistics

KW - quantiles

KW - acceptability

KW - rating distribution

U2 - 10.1007/s41233-016-0002-1

DO - 10.1007/s41233-016-0002-1

M3 - Article

VL - 1

JO - Quality and User Experience

JF - Quality and User Experience

SN - 2366-0139

IS - 1

ER -