Research conducted in the scope of the MUEXLab focuses on three main areas:
- Modeling and assessment of QoE and the user experience,
- Monitoring and measuring QoE,
- Management and control of QoE.
We focus on the identification of the key influence factors impacting QoE, taking into account user, system, and context related factors. Research is conducted using quality assessment techniques involving the collection of both subjective and objective metrics. Of particular interest are emerging immersive services, such as AR/VR communications, 360-degree video streaming, networked games, and multiparty audiovisual communication services. Understanding and modeling the factors that impact QoE inherently calls for a multidisciplinary approach, combining knowledge from engineering, psychology, and user interface design. The modeling process involves using both analytical approaches and “black box” approaches based on various machine learning techniques.
Selected research results:
Modeling QoE for cloud gaming
Cloud gaming has been recognized as a promising shift in the online game industry, with the aim being to deliver high-quality graphics games to any type of end user device. The concepts of cloud computing are leveraged to render the game scene as a video stream which is then delivered to players in real-time. Given high bandwidth and strict latency requirements, a key challenge faced by cloud game providers lies in configuring the video encoding parameters so as to maximize player QoE while meeting bandwidth availability constraints. We conduct subjective studies aimed at identifying QoE-driven video encoding adaptation strategies. Empirical results are used to derive analytical QoE estimation models as functions of bitrate and framerate, while also taking into account game type and player skill. Given that results indicate that different QoE-driven video adaptation policies should likely be applied for different types of games, we further study objective video metrics that may be used to classify games for the purpose of choosing an appropriate and QoE-driven video codec configuration strategy.
- I. Slivar, M. Sužnjević, L. Skorin-Kapov, "Game Categorization for Deriving QoE-Driven Video Encoding Configuration Strategies for Cloud Gaming", ACM Transactions on Multimedia Computing, Communications, and Applications (ACM TOMM), vol. 14, issue 3, June 2018.
- I. Slivar, L. Skorin-Kapov, M. Sužnjević, "Cloud Gaming QoE Models for Deriving Video Coding Adaptation Strategies", Proc. of ACM Multimedia Systems (MMSys'16), Klagenfurt, Austria, 2016.
- M. Sužnjević, I. Slivar, L. Skorin-Kapov, "Analysis and QoE Evaluation of Cloud Gaming Service Adaptation Under Different Network Conditions: the Case of NVIDIA GeForce NOW", in proc. of 8th International Conf. on Quality of Multimedia Experience (QoMEX 2016), Lisbon, Portugal, 2016
Immersive VR and AR services
Advances in the field of VR have focused on improvements in VR interaction techniques, including support for different locomotion, or navigation, methods. We study how different navigation methods influence the user perception of immersion and the occurrence of simulator (or VR) sickness. Going beyond subjective assessment methods, we further collect physiological measures (e.g., using EEG) to explore correlations with simulator sickness. With respect to multiuser networked VR environments, we explore the impact of various system and network QoS factors on user QoE, in scenarious involving VR-based games and collaborative virtual environments.
In the domain of AR, we are cooperating with the company DivIT in the scope of the ARIEN project. Our focus is on assessing the impact of different compression techniques on subjective quality in holoportation-like scenarios, involving real-time interactive communication and collaboration between two or more participants.
- S. Vlahović, M. Sužnjević, and L. Skorin-Kapov. "Challenges in Assessing Network Latency Impact on QoE and In-Game Performance in VR First Person Shooter Games.", 15th International Conference on Telecommunications (ConTEL), Graz, Austria, July 2019
- S. Vlahović, M. Sužnjević, and L. Skorin-Kapov. "The Impact of Network Latency on Gaming QoE for an FPS VR Game.", 11th International Conference on Quality of Multimedia Experience (QoMEX 2019), Berlin, Germany, June 2019
- S. Vlahović, M. Sužnjević, and L. Skorin-Kapov. "Subjective Assessment of Different Locomotion Techniques in Virtual Reality Environments." 10th International Conference on Quality of Multimedia Experience (QoMEX 2018), Sardinia, Italy, May 2018.
WebRTC-based mobile multiparty audiovisual telemeetings
Managing interactive and multiparty video conferencing services requires an understanding of the key underlying QoE influence factors. A key challenge faced by service providers lies in configuring the video encoding parameters so as to maximize participant QoE while meeting resource (network and mobile device) availability constraints. Currently developed QoE models can for the most part be applied to two interlocutors and in desktop environments. However, there is a lack of studies that focus on modelling and optimizing QoE for such services in a mobile context, and in particular in the case of multiparty scenarios. Multiparty video call optimization is thus a challenging task due to dynamic wireless networks, heterogeneous mobile devices, and contexts. Hence, there is a need to investigate thresholds within video encoding parameters that can be used to determine optimal adaptation strategies for mobile multiparty video conferencing services. We study how to specify QoE-driven service adaptation for mobile multiparty video conferencing calls under variable resource availability. Extensive subjective user tests are conducted to identify key QoE influence factors and provide input for deriving QoE models.
- D. Vučić, L. Skorin-Kapov, "QoE Assessment of Mobile Multiparty Audiovisual Telemeetings", IEEE Access, Vol. 8, 2020, DOI: 10.1109/ACCESS.2020.3000467
- D. Vučić, L. Skorin-Kapov, "The Impact of Packet Loss and Google Congestion Control on QoE for WebRTC-based Mobile Multiparty Audiovisual Telemeetings", in Proc. of the 25th International Conference on MultiMedia Modeling (MMM 2019), Thessaloniki, Greece, Jan. 2019
- D. Vučić, L. Skorin-Kapov, M. Sužnjević, "The impact of bandwidth limitations and video resolution size on QoE for WebRTC-based mobile multi-party video conferencing", in Proc. of the 5th ISCA/DEGA Workshop on Perceptual Quality of Systems, PQS, Aug. 2016, Berlin
- D. Vučić, L. Skorin-Kapov. "The Impact of Mobile Device Factors on QoE for Multi-Party Video Conferencing via WebRTC", Proc. of the 13th International Conference on Telecommunications, ConTEL, Graz, Austria, July 2015.
Multidimensional analysis of Web browsing QoE
The increasing use of mobile devices for Web browsing has driven a rising interest in understanding and enhancing the user experience when accessing mobile Web sites. We study multiple factors that impact the quality of the user experience while browsing the mobile Web, focusing on design, information quality, and loading time. Based on the results of subjective studies, we model QoE in the context of browsing information, thematic, and e-mail portals via browsers on smartphones and tablet devices. Results have shown the existence of significant effects of examined factors on QoE, while mutual relations between QoE and multiple perceivable characteristics of the experience that contribute to its quality, referred to as QoE features, are modelled, identifying the importance of the distinct dimensions in terms of overall user perceived QoE.
- S. Baraković, L. Skorin-Kapov, "Survey of Research on Quality of Experience Modelling for Web Browsing", in Quality and User Experience, Springer, available online July 2017.
- S. Baraković, L. Skorin-Kapov, "Modeling the relationship between design/performance factors and perceptual features contributing to Quality of Experience for mobile Web browsing", Computers in Human Behavior, Vol 74, Sept 2017 (available online April 2017)
- S. Baraković, L. Skorin-Kapov, "Multidimensional Modelling of Quality of Experience for Mobile Web Browsing", Computers in Human Behavior, Sept. 2015, Vol. 50
- M. Varela, L. Skorin-Kapov, T. Mäki and T. Hoßfeld. "QoE in the Web: A Dance of Design and Performance", in Proceedings of the 7th International Workshop on Quality of Multimedia Experience (QoMEX 2015), Costa Navarino, Greece, May 2015.
Putting to use QoE estimation models, both white box and black box, requires the collection of necessary input parameters for various types of services. Monitoring architectures commonly collect data using passive or active probes deployed along various parts of the service delivery chain. Data may be collected at different levels, including network-level QoS measurements (e.g., packet loss, delay, jitter) or traffic characteristics, as well as application-level measurements (e.g., buffer status, audio/video bitrate, resolution, and framerate). Moreover, data may be considered at different aggregation levels and collected at different time scales to facilitate either monitoring and control in (near) real-time, or subsequent to service usage. A complex task lies in identifying the root causes of QoE impairments, as such causes may occur in different regions of the network.
Selected research results:
QoE and KPI monitoring of encrypted video traffic using machine learning techniques
Solutions for in-network monitoring of QoE-related KPIs are a necessary prerequisite to detecting potential impairments, identifying their root cause, and consequently invoking QoE-aware management actions. In cooperation with the company Ericsson Nikola Tesla, we are studying how to apply machine learning techniques to train QoE and KPI classifiers for adaptive video streaming using features extracted from encrypted traffic. We have developed tools for collecting ground truth application layer data (primarily focused on YouTube), and have collected numerous datasets in both lab and mobile network environments to train and analyze various ML-based models.
- I. Oršolić, L. Skorin-Kapov, T. Hoßfeld, "To Share or Not to Share? How Exploitation of Context Data Can Improve In-Network QoE Monitoring of Encrypted YouTube Streams.", 11th International Conference on Quality of Multimedia Experience (QoMEX 2019), Berlin, Germany, June 2019
- I. Bartolec, I. Oršolić, L. Skorin-Kapov, "In-Network QoE and KPI Monitoring of Mobile YouTube Traffic: Insights for Encrypted iOS Flows", 11th International Conference on Quality of Multimedia Experience (QoMEX 2019), Berlin, Germany, June 2019
- I. Oršolić, P. Rebernjak, M. Sužnjević, L. Skorin-Kapov, "In-Network QoE and KPI Monitoring of Mobile YouTube Traffic: Insights for Encrypted iOS Flows", 14th International Conference on Network and Service Management, CNSM 2018, Rome, Italy, Nov. 2018
- I. Oršolić, M. Sužnjević, L. Skorin-Kapov. "YouTube QoE Estimation from Encrypted Traffic: Comparison of Test Methodologies and Machine Learning Based Models", 10th International Conference on Quality of Multimedia Experience (QoMEX 2018), Sardinia, Italy, May 2018
- I. Orsolic, D. Pevec, M. Sužnjević, L. Skorin-Kapov, "A Machine Learning Approach to Classifying YouTube QoE Based on Encrypted Network Traffic", Multimedia Tools and Applications, Springer, 2017
A significant amount of research has been devoted to understanding, measuring, and modeling QoE for a variety of media services. The next logical step is to actively exploit that accumulated knowledge to improve and manage the quality of multimedia services, while at the same time ensuring efficient and cost-effective network operations. Moreover, with many different players involved in the end-to-end service delivery chain, identifying the root causes of QoE impairments and finding effective solutions for meeting the end users’ requirements and expectations in terms of service quality is a challenging and complex problem. The goal of QoE management may be related to optimizing end-user QoE (end-users' perspective), while making efficient (current and future) use of network/system resources and maintaining a satisfied customer base (providers' perspective). Of particular interest is the exploitation of AI and ML techniques in the context of managing QoE for advanced interactive and immersive multimedia services provisioned over 5G networks.
Selected research results:
Different metrics and optimization objectives driving QoE management
User-centric service and application management focuses on QoE as perceived by the end user. Thereby, the goal is to maximize QoE while ensuring fairness among users, e.g., for resource allocation and scheduling in shared systems. We propose the definition of a novel generic QoE fairness index F, and argue that neither QoS fairness nor Jain’s fairness index meet all of the desirable QoE-relevant properties which are met by F. Consequently, the proposed index F may be used to compare QoE fairness across systems and applications, thus serving as a benchmark for QoE management mechanisms and system optimization. We further study how the choice of different QoE metrics, the importance of fairness, and the variations between users, can affect optimal QoE management choices for service providers.
- T. Hoßfeld, P.E. Heegaard, L. Skorin-Kapov, M. Varela. "Deriving QoE in systems: from fundamental relationships to a QoE-based Service-level Quality Index". Quality and User Experience 5, 7 (2020). https://doi.org/10.1007/s41233-020-00035-0
- L. Skorin-Kapov, M. Varela, T. Hoßfeld, K-T. Chen, "A Survey of Emerging Concepts and Challenges for QoE Management of Multimedia Services", ACM Transactions on Multimedia Computing, Communications, and Applications (ACM TOMM), vol 14, issue 2, May 2018. (online appendix to paper available here)
- T. Hoßfeld, L. Skorin-Kapov , P. Heegaard, M. Varela, "A New QoE Fairness Index for QoE Management", Quality and User Experience, Springer, 2018, open access: https://doi.org/10.1007/s41233-018-0017-x.
- T. Hoßfeld, L. Skorin-Kapov , P. Heegaard, M. Varela, Definition of QoE Fairness in Shared Systems, IEEE Communication Letters, Vol. 21, Issue 1, Jan. 2017, available online, DOI: 10.1109/LCOMM.2016.2616342
- T. Hoßfeld, P. Heegaard, L. Skorin-Kapov and M. Varela. "No Silver Bullet: QoE Metrics, QoE Fairness, and User Diversity in the Context of QoE Management", in proc. of 9th International Conference on Quality of Multimedia Experience (QoMEX 2017), Erfurt, Germany, May 2017.
- C. Moldovan, L. Skorin-Kapov, P. Heegaard, T. Hoßfeld;, "Optimal Fairness and Quality in Video Streaming With Multiple Users", Proc. of ITC 30 (International Teletraffic Congress), Sept. 2018, Vienna, Austria.
Cross-layer QoE management
We study cross-layer approaches where application-level user- and service-related knowledge is utilized for various scenarious, such as QoE-driven admission control, resource allocation, and path optimization. We have proposed a Quality Matching and Optimization Function Application Server (QMO AS) which calculates feasible service configurations based on application-specific parameters, providing both an optimal configuration, and several sub-optimal ones (resulting in a so-called Media Degradation Path, or MDP). The MDP specifies a mapping between session parameters, resource requirements, and corresponding user experience quality levels. The QMO AS is included in the network along the signaling path and interfaces with policy and resource allocation entities to support QoE-driven resource management.
- K. Ivešić, L. Skorin-Kapov, M. Matijašević. "Cross-layer QoE-driven Admission Control and Resource Allocation for Adaptive Multimedia Services in LTE", Journal of Network and Computer Applications, Elsevier, doi:10.1016/j.jnca.2014.09.010, available online Sept. 2014
- R. Schatz, M. Fiedler and L. Skorin-Kapov. "QoE-based Network and Application Management". in "Quality of Experience: Advanced Concepts, Applications and Methods", T-Labs Series in Telecommunication Services, S. Möller and A. Raake (eds.), Springer, March 2014, pp. 411-426.
- O. Dobrijevic, A. Kassler, L. Skorin-Kapov, M. Matijasevic. "Q-POINT: QoE-Driven Path Optimization Model for Multimedia Services", 12th Intl Conf. on Wired and Wireless Internet Communications, in Lecture Notes in Computer Science, vol. 8458, Springer, pp. 134-147, 2014.
- L. Skorin-Kapov, K. Ivesic, G. Aristomenopoulos and S. Papavassiliou, "Approaches for Utility-Based QoE-Driven Optimization of Network Resource Allocation for Multimedia Services", in "Data Traffic Monitoring and Analysis: From Measurement, Classification and Anomaly Detection to Quality of Experience", Lecture Notes in Computer Science, Vol. 7754, Springer Computer Communications and Networks series, 2013, pp. 337-358
- A. Kassler, L. Skorin-Kapov, O. Dobrijevic, M. Matijasevic, P. Dely. "Towards QoE-driven Multimedia Service Negotiation and Path Optimization with Software Defined Networking", Proc. of IEEE SoftCOM, Sept. 2012.