Putting to use QoE estimation models, both white box and black box, requires the collection of necessary input parameters for various types of services. Monitoring architectures commonly collect data using passive or active probes deployed along various parts of the service delivery chain. Data may be collected at different levels, including network-level QoS measurements (e.g., packet loss, delay, jitter) or traffic characteristics, as well as application-level measurements (e.g., buffer status, audio/video bitrate, resolution, and framerate). Moreover, data may be considered at different aggregation levels and collected at different time scales to facilitate either monitoring and control in (near) real-time, or subsequent to service usage. A complex task lies in identifying the root causes of QoE impairments, as such causes may occur in different regions of the network.

QoE and KPI monitoring of encrypted video traffic using machine learning techniques

Solutions for in-network monitoring of QoE-related KPIs are a necessary prerequisite to detecting potential impairments, identifying their root cause, and consequently invoking QoE-aware management actions. In cooperation with the company Ericsson Nikola Tesla, we are studying how to apply machine learning techniques to train QoE and KPI classifiers for adaptive video streaming using features extracted from encrypted traffic. We have developed tools for collecting ground truth application layer data (primarily focused on YouTube), and have collected numerous datasets in both lab and mobile network environments to train and analyze various ML-based models.

Selected publications:

• I. Oršolić, L. Skorin-Kapov, T. Hoßfeld, "To Share or Not to Share? How Exploitation of Context Data Can Improve In-Network QoE Monitoring of Encrypted YouTube Streams.", 11th International Conference on Quality of Multimedia Experience (QoMEX 2019), Berlin, Germany, June 2019
• I. Oršolić, P. Rebernjak, M. Sužnjević, L. Skorin-Kapov, "In-Network QoE and KPI Monitoring of Mobile YouTube Traffic: Insights for Encrypted iOS Flows", 14th International Conference on Network and Service Management, CNSM 2018, Rome, Italy, Nov. 2018
• I. Oršolić, M. Sužnjević, L. Skorin-Kapov. "YouTube QoE Estimation from Encrypted Traffic: Comparison of Test Methodologies and Machine Learning Based Models", 10th International Conference on Quality of Multimedia Experience (QoMEX 2018), Sardinia, Italy, May 2018
• I. Orsolic, D. Pevec, M. Sužnjević, L. Skorin-Kapov, "A Machine Learning Approach to Classifying YouTube QoE Based on Encrypted Network Traffic", Multimedia Tools and Applications, Springer, 2017

KPI monitoring of encrypted video traffic considering user playback interactions

Although viewers frequently interact with the video player, such user interactions are rarely accounted for in data collection and QoE/KPI monitoring models. User interactions impact traffic patterns and, consequently, the performance of machine learning (ML) models that rely on features derived from traffic. Within the scope of the Q-MERSIVE project, we investigate common user playback interactions (e.g., pause, seeking, abandonment) for popular video streaming services, develop models of user interactions, and incorporate these models into the data collection procedure in order to generate realistic datasets on which ML models will be trained. 

Selected publications:

• I. Bartolec, I. Oršolić, L. Skorin-Kapov, "Impact of User Playback Interactions on In-Network Estimation of Video Streaming Performance.", in IEEE Transactions on Network and Service Management, 2022
• I. Bartolec, I. Oršolić, L. Skorin-Kapov, "Inclusion of End User Playback-Related Interactions in YouTube Video Data Collection and ML-Based Performance Model Training.", 12th International Conference on Quality of Multimedia Experience (QoMEX), 2020