Crowdsourcing Quality of Experience Experiments

• Published in: Evaluation in the Crowd. Crowdsourcing and Human-Centered Experiments Vol. 10264; pp. 154 - 190
• Main Authors: Egger-Lampl, Sebastian, Redi, Judith, Hoßfeld, Tobias, Hirth, Matthias, Möller, Sebastian, Naderi, Babak, Keimel, Christian, Saupe, Dietmar
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2017; Springer International Publishing
• Series: Lecture Notes in Computer Science
• ISBN: 3319664344; 9783319664347
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-66435-4_7

Crowdsourcing enables new possibilities for QoE evaluation by moving the evaluation task from the traditional laboratory environment into the Internet, allowing researchers to easily access a global pool of workers for the evaluation task. This makes it not only possible to include a more diverse population and real-life environments into the evaluation, but also reduces the turn-around time and increases the number of subjects participating in an evaluation campaign significantly, thereby circumventing bottle-necks in traditional laboratory setups. In order to utilise these advantages, the differences between laboratory-based and crowd-based QoE evaluation are discussed in this chapter.