DATASETS AND TOOLS

All of the below listed datasets are made publicly available for research purposes. If you use the datasets, we kindly ask that you cite the relevant publication that is listed. 

 

Dataset description

 

Corresponding publications

(please cite if using the dataset)

Dataset: FPV Drone Control QoE - Quality scores of a subjective study Phase 2
(download here)
The findings presented in this dataset are based on two separate subjective studies, referred to as Study 1 and Study 2. The first study was conducted at the University of Zagreb, Faculty of Electrical Engineering and Computing in January of 2021, while the second study was conducted six months later at the University Center of Defence (CUD), San Javier Air Force Base, Spain. A detailed description of the test methodology is given in the corresponding publication.

M. Šilić; M. Sužnjević; L. Skorin-Kapov; N. Skorin-Kapov; I. Lorenzana, Marcelo: The Impact of Video Encoding Parameters on QoE of Simulated FPV Drone Control, Multimedia Tools and Applications, 2024, 1-33. DOI: https://doi.org/10.1007/s11042-024-18442-2

Competitive Multiplayer VR Gaming: Anonymized Dataset of Player Responses

(request access here)

Contained within this repository is a comprehensive collection of anonymized player responses obtained from a user study (N = 32) investigating the influence of various factors—including the game itself, social dynamics, network access, network latency, and player expertise—on gaming experiences within multiplayer virtual reality. The study focused on perceived network quality, social interaction, competitive feelings, and overall QoE (Quality of Experience). It employed Meta Quest head-mounted displays paired with handheld controllers, showcasing two distinct competitive games: Eleven Table Tennis and Blaston.

S. Vlahovic, I. Slivar, M. Silic, L. Skorin-Kapov, M. Suznjevic. "Exploring the Facets of the Multiplayer VR Gaming Experience", ACM Transactions on Multimedia Computing, Communications, and Applications, 2024

Video streaming datasets: Improving the Transfer of Machine Learning-Based Video QoE Estimation Across Diverse Networks

(download here)

The datasets in this repository consist of video on demand streaming data collected at two locations (Würzburg, Germany and Zagreb, Croatia) and across two years (2020 and 2021). We refer to the datasets by using the following labels: Wue_2020, Wue_2021, Zag_2020, Zag_2021. The data includes network traffic features used to estimate Quality of Experience (QoE) and Key Performance Indicators (KPI) of video streaming sessions using machine learning. The traffic features are annotated with QoE/KPI classes, with samples considered both on a session-level (per-video) and in real-time fashion (per-second). The datasets are collected for and presented in the journal article entitled "Improving the Transfer of Machine Learning-Based Video QoE Estimation Across Diverse Networks", authored by Michael Seufert and Irena Oršolić, published in IEEE Transactions on Network and Service Management in 2023

Michael Seufert, Irena Oršolić, "Improving the Transfer of Machine Learning-Based Video QoE Estimation Across Diverse Networks", IEEE Transactions on Network and Service Management, 2023

Questionnaire-based survey investigating the influence of various system-related factors on overall experience and quality perception of audiovisual calls on smartphones (download here).

This dataset contains the results of two surveys: Survey 1 -  272 participants, conducted in February 2020;  Survey 2 - 249 participants, conducted in October 2021. Data was collected on user opinions regarding the influence of various factors related to media quality, functional support of the service, usability, service design, and resource consumption. The focus was on audiovisual calls established in a leisure context, as opposed to business related calls/meetings.

D. Vučić, S. Baraković, L. Skorin-Kapov, Survey on user perceived system factors influencing the QoE of audiovisual calls on smartphones. Multimedia Tools and Applications (2022). https://doi.org/10.1007/s11042-022-14173-4

OTT video streaming dataset containing user interactions (download here)

The dataset contains network traffic statistics and ground-truth application-layer data corresponding to 7424 video sessions from the global OTT video streaming provider. Data was collected over a four-month period, from December 2020 to March 2021. In total, 2030 videos had no user interactions executed during playback, 1749 videos were paused once during playback, 1808 videos were seeked forward at one point during playback, and 1837 videos were terminated before the video ended.

I. Bartolec, I. Orsolic, and L. Skorin-Kapov. "Impact of User Playback Interactions on In-Network Estimation of Video Streaming Performance", IEEE Transactions on Network and Service Management

CGD: A Cloud Gaming Dataset with Gameplay Video and Network Recordings (download here)

CGD, a dataset consisting of 600 game streaming sessions corresponding to 10 games of different genres being played and streamed using the following encoding parameters: bitrate (5, 10, 20 Mbps), resolution (720p, 1080p), and frame rate (30, 60 fps). For every combination repeated five times for each game, the dataset includes: 1) gameplay video recordings, 2) network traffic traces, 3) user input logs (mouse and keyboard), and 4) streaming performance logs.

I. Slivar, K. Bacic, I. Orsolic, L. Skorin-Kapov, and M. Suznjevic. "CGD: A Cloud Gaming Dataset with Gameplay Video and
Network Recordings",
 In 13th ACM Multimedia Systems Conference (MMSys ’22), Athlone, Ireland, June 14–17, 2022. 

 

FPV dataset - subjective user scores of QoE, graphics quality, fluidity, and willingness to continue using the system (download here)

We have performed a subjective study on using the Orqa FPV.SkyDive drone flight simulator with FPV goggles gathering over 250 responses on various subjective QoE metrics from 14 participants. The dataset is analyzed and described in the corresponding paper.

M. Šilić, M. Sužnjević, and L. Skorin-Kapov. "QoE Assessment of FPV Drone Control in a Cloud Gaming Based Simulation", 13th International Conference on Quality of Multimedia Experience (QoMEX 2021), Montreal, Canada, June 2021.

YouTube QoE/KPI classification with user interactions - network traffic features annotated with MOS/KPIs (request access here)

Six datasets were collected during the last months of 2019, and contain sessions with none, or one manually triggered user interaction (e.g., pause, seek, abandon, playback speed), with one dataset containing a pair, or a combination of said interactions. Each dataset contains YouTube video streaming sessions that were collected on an Android smartphone, and native YouTube application. A total of 17 machine learning-based models for per-video KPI classification were trained on various combinations of those datasets. A detailed list of features, selected features per model and feature importances are provided here.

I. Bartolec, I. Orsolic, L. Skorin-Kapov, "Inclusion of End User Playback-Related Interactions in YouTube Video Data Collection and ML-Based Performance Model Training", 12th International Conference on Quality of Multimedia Experience (QoMEX), June 2020.

YouTube per-video and per-second QoE/KPI classification (request access here)

Two datasets collected in 2019 corresponding to the streaming of 400 YouTube videos on (1) Android and (2) iOS platform. Both datasets include per-video network traffic features, as specified in the corresponding publication, and MOS/KPI labels. Moreover, the Android dataset is available in the format appropriate for real-time (per-second) KPI classification, where 1s intervals are denoted with a set of network traffic features and labelled with KPIs.

I. Orsolic, L. Skorin-Kapov, "A Framework for In-Network QoE Monitoring of Encrypted Video Streaming", IEEE Access, vol. 8, pp. 74691-74706, 2020, DOI: 10.1109/ACCESS.2020.2988735 

YouTube QoE/KPI classification with user interactions - network traffic features annotated with MOS/KPIs (request access here)

Two datasets involving YouTube video streaming sessions to an Android smartphone were collected during Feb. and Mar. of 2019. All traffic was transmitted using the QUIC protocol. Dataset D1 was recorded by streaming 10 different YouTube playlists (each with 10 videos) without any user interactions invoked. Dataset D2 was recorded by streaming one playlist consisting of 8 videos, and included the previously described user interactions, manually invoked during playback. Each dataset is a CSV file used to train machine learning-based models. The file contains all derived features (on a per video level) and the following target labels: longest resolution (two classes); initial delay (two classes), average video bitrate (two and three classes). The lab set up and list of features selected for machine learning model training are provided here.

Ivan Bartolec, Irena Oršolić, Lea Skorin-Kapov, "In-Network YouTube Performance Estimation in Light of End User Playback-Related Interactions", 11th International Conference on Quality of Multimedia Experience, QoMEX 2019, Berlin, Germany.

YouTube per-video QoE/KPI classification (request access here)

We have collected network- and application-level data from 394 YouTube video streaming sessions on Android device connected to a WiFi network in order to train machine learning models which predict YouTube performance from network traffic features. The dataset is described and analysed in the paper.

I. Oršolić, M. Sužnjević, L. Skorin-Kapov. "YouTube QoE Estimation from Encrypted Traffic: Comparison of Test Methodologies and Machine Learning Based Models", 10th International Conference on Quality of Multimedia Experience (QoMEX 2018), Sardinia, Italy, May 2018.

YouTube per-video QoE/KPI classification (request access here)

A similar dataset was collected with an iOS client, described and analysed in the paper

I. Oršolić, P. Rebernjak, M. Sužnjević, L. Skorin-Kapov, "In-Network QoE and KPI Monitoring of Mobile YouTube Traffic: Insights for Encrypted iOS Flows", in proc. of the 14th International Conference on Network and Service Management, CNSM 2018, Rome, Italy, Nov. 2018.

MMORPG QoE gaming dataset - subjective user scores of QoE, immersion, fluidity, responsiveness, immersion and challenge level (request access here)

We have performed two subjective studies on MMORPG World of Warcraft gathering over 10 000 responses on various subjective metrics from 104 participants. The dataset is described and analyzed in the corresponding papers.

M. Suznjevic, L. Skorin-Kapov, Aleksandra Cerekovic, Maja Matijasevic, "How to Measure and Model QoE for Networked Games?  The case study of World of Warcraft ", Multimedia Systems, Springer, 2019.

M. Sužnjevic, L. Skorin-Kapov, M. Matijašević. "The Impact of User, System, and Context factors on Gaming QoE: a Case Study Involving MMORPGs", Proc. of NetGames 2013, Denver, USA, Dec. 9-10, Dec. 2013.

Cloud gaming dataset - PC game play video traces annotated with video metrics (download here)

We have recorded gaming sessions of 25 different video games and collected 225 different video traces. This data set is described and analyzed in the corresponding paper.

I. Slivar, M. Sužnjević, L. Skorin-Kapov, "Game Categorization for Deriving QoE-Driven Video Encoding Configuration Strategies for Cloud Gaming", ACM Transactions on Multimedia Computing, Communications, and Applications (ACM TOMM), vol. 14, issue 3, June 2018.

Cloud gaming dataset - subjective user scores of QoE, graphics quality, fluidity and willingness to play (request access here)

We have performed two subjective studies on 3 different games gathering over 8 000 responses on various subjective metrics from 80 participants. The dataset is analysed and described in the corresponding paper.

I. Slivar, M. Sužnjević, L. Skorin-Kapov, "Game Categorization for Deriving QoE-Driven Video Encoding Configuration Strategies for Cloud Gaming", ACM Transactions on Multimedia Computing, Communications, and Applications (ACM TOMM), vol. 14, issue 3, June 2018.