The Multimedia Quality of Experience Research Lab (MUEXlab) is a research lab at the Department of Telecommunications, Faculty of Electrical Engineering and Computing, University of Zagreb. The lab conducts multidisciplinary research in the areas of User Experience (UX) and Quality of Experience (QoE) modeling and management of advanced multimedia services, such as multiparty audiovisual communications, video streaming, virtual/augmented reality, and networked games.
The Multimedia Quality of Experience Research Lab (MUEXlab).
News
On Nov. 14, 2024, Professor Lea Skorin-Kapov visited the Grupo de Tratamiento de Imágenes (GTI) at Universidad Politécnica de Madrid (UPM) and gave a research talk on “Understanding and Managing Quality of Experience in the Era of Networked Multimedia Services”. A big thank you to Professor Narciso García, Associate Professor Jesús Gutiérrez Sánchez, and the whole GTI team for organizing the talk and for the great opportunity to visit the GTI lab, during which the GTI team presented a number of inspiring demos showcasing innovative XR service design, application domains, and research challenges. During her stay in Madrid, prof. Skorin-Kapov also had the opportunity to visit Nokia XR Lab, hosted by lead scientist dr. Pablo Pérez, where the team presented cutting-edge demos and discussed applications of XR, AI, and 5G/6G networks for supporting future human communication and collaboration.
Members of MUEXlab, doctoral student Katarina Mišura and Professor Mirko Sužnjević, participated in the NATO Science & Technology Organization (STO) Modelling and Simulations Group (MSG) Symposium on “Modelling and Simulation as enabler for Digital Transformation in NATO and Nations” (STO-MP-MSG-217), held in La Spezia, Italy.
At the symposium, Katarina delivered a full paper presentation titled "Extended Reality User Interface Quality of Experience Testing for Military Purposes Using a Video Game", co-authored with Miro Čolić and Mirko Sužnjević, during the session on VR/XR Technology and Applications. The paper can be accessed here. Additionally, participants had the opportunity to take part in a technical visit to the NATO STO Centre for Maritime Research and Experimentation (CMRE) in La Spezia, Italy.
MUEXlab doctoral students, Lovro Boban, Katarina Mišura, Mirta Moslavac, and Nikolina Roso, attended the PhD Forum event, organized as part of the 6th International Conference on Smart Systems and Technologies (SST 2024) held in Osijek, Croatia.
During the event, the students presented the results of their research through short presentations and a poster session, covering the following topics:
- Lovro Boban - "Procedural Generation of Safe Falling Animations of a 3D Character Using Reinforcement Learning"
- Katarina Mišura - "QoE Assessment for Geo-located Augmented Reality"
- Mirta Moslavac - "Quality of Experience and User Experience in Collaborative Social Extended Reality" (awarded third place for Best Pitch Presentation)
- Nikolina Roso - "Impact of Video and Network Parameters on Teleoperation: Current Limitations and Opportunities"
Additionally, members of MUEXlab, including Professor Maja Matijašević as a member of the Steering Committee and Sara Vlahović as a member of the Program & Organizing Committee, contributed to the organization of the event.
We congratulate our students on successfully presenting their work, and a special congratulations to Mirta Moslavac for winning the award!
Members of MUEXlab participated in the 32nd International Conference on Software, Telecommunications, and Computer Networks (SoftCOM 2024), held in Bol, Croatia.
At the conference, MUEXlab researchers showcased their work through several presentations. In the Special Session on Next Generation Wired and Wireless Networks II, Katarina Mišura delivered a full paper presentation titled "Augmented Reality Interface Quality of Experience Assessment Using Video Game Simulation," co-authored by Katarina Mišura and Mirko Sužnjević. Additionally, during the Poster/Abstract session, Professor Mirko Sužnjević presented a poster titled "Determination of the Impact of Video Encoding Parameters on QoE Using Passive Viewing Tests," co-authored by Dominik Arih, Ilija Barišić and Mirko Sužnjević.
Activities are well under way in the MUEXlab and IoTlab in the scope of the project DIGIPHY - XR Communication and Interaction Through a Dynamically Updated Digital Twin of a Smart Space, funded by the EU via the National Recovery and Resilience Plan (NPOO 2021-2026). Project partners include the University of Zagreb Faculty of Electrical Engineering and Computing (coordinator), Ericsson Nikola Tesla, and Diversitas IT Sustavi. The project will run from 02/2024 – 06/2026. Welcome to new MUEXlab team members Lea Brzica and Filip Matanović who have joined the project! More updates to follow soon!
In July 2023, the University Carlos III of Madrid (UC3M) in Spain hosted the 2nd International Summer School on eXtended Reality Technology and eXperience (XRTX). This four-day event brought together PhD students to share their research and participate in both theoretical and practical sessions on XR technology. Our PhD student, Mirta Moslavac, attended the school and presented a poster titled "Measuring and Modelling Quality of Experience and User Experience in Collaborative Social XR".
During the practical sessions, participants were divided into groups to design and prototype a VR archery game. The project titled "Draw to the StaVRs", which Mirta contributed to, was recognized with the award for "Best Project in Design Definition".
We are pleased to share an insightful interview with Mr. Konstantinos Grapsas, the EPIIC Project Officer from the European Commission. In this video, Mr. Grapsas offers valuable perspectives on the EPIIC project and the crucial role of the European Defence Fund (EDF) in advancing military aviation technology. The LinkedIn post is available here, and the full interview is available at the link.
Several members of MUEXlab attended the 16th International Conference on Quality of Multimedia Experience (QoMEX'24) in Karlshamn, Sweden. Professor Skorin-Kapov, head of MUEXlab, served as a general co-chair of the conference.
At the co-located workshop on measuring and modeling user experiences in XR, Mirta Moslavac and Sara Vlahović delivered short pitch presentations. Following the workshop, MUEXlab members presented their accepted papers at the conference. Mirta Moslavac presented a full paper entitled "Evaluating QoE in Collaborative Co-Located Training Using Heterogeneous AR Devices", co-authored by Mirta Moslavac, Sara Vlahović, and Lea Skorin-Kapov. Nikolina Roso presented a short paper entitled "Exploring Conversations with AI NPCs: The Impact of Token Latency on QoE and Player Experience in a Text-Based Game", co-authored by Nikolina Roso, Sara Vlahović, Nenad Markuš, and Mirko Sužnjević, with Nenad Markuš joining the poster presentation.
Additionally, as one of the Diversity and Inclusion chairs, Sara Vlahović helped organize and moderate a panel discussion on the prospective evolution of QoMEX. Professor Mirko Sužnjević participated as a panelist, contributing his insights on the topic.
We are pleased to share that an article titled "Not just cybersickness: short‑term efects of popular VR game mechanics on physical discomfort and reaction time" authored by Sara Vlahovic, Lea Skorin-Kapov, Mirko Sužnjević, and Nina Pavlin‑Bernardic, has been published in the esteemed journal Virtual Reality (VIRE; Springer).
This article presents the results of two user studies aimed at evaluating the intensity of VR-induced symptoms and effects that occur during VR gaming, using both subjective and objective measures. The article can be accessed here.
On Saturday, April 27, 2024, elementary school students visited FER to explore our laboratory's activities as part of the RADDAR program, which offers maths and computer science workshops for children in grades 5 through 8. Professor Lea Skorin-Kapov, the head of MUEXlab, delivered a brief presentation on the Quality of Experience (QoE) of immersive services. Following this, MUEXlab members Lovro Boban, Ana Jović, Mirta Moslavac, and Sara Vlahović guided the students through trying out various virtual and augmented reality platforms. To conclude the workshop, students conducted their own QoE assessments of the showcased applications.
We are delighted to announce the publication of an article entitled "The impact of video encoding parameters on QoE of simulated FPV drone control" authored by Matko Šilić, Mirko Sužnjević, Lea Skorin-Kapov, Nina Skorin-Kapov, and Marcelo Izquierdo Lorenzana in the esteemed journal Multimedia Tools and Applications (MTAP; Springer).
This article presents the results of two user studies aimed at assessing the Quality of Experience (QoE) of a First-person View (FPV) system integrated with a cloud-based drone flight simulator. The research, conducted in collaboration with the Spanish Air Force Academy, provides valuable insights into the relationship between video encoding parameters and user experience in FPV drone control. The article can be accessed here.
Our PhD student Mirta Moslavac participated in the 4th NOVsky Art&Science Festival on September 15, 2023, in Novska, as a part of FER’s attendance at the festival. Drawing an audience of over 3000 participants spanning all ages, attendees had the opportunity to be introduced to the realms of augmented and mixed reality, providing them with the chance to actively explore and engage with these immersive technologies.
(image source: Novska.IN portal)
Following his participation in the 11th Network Traffic Measurement and Analysis (TMA) PhD school (Napoli, Italy), our PhD student Ivan Bartolec received the Best Poster Award for his poster entitled "Performance estimation of encrypted video streaming considering end-user playback-related interactions". Congratulations, Ivan!
Professor Lea Skorin-Kapov presented the short paper "An Initiative Toward an Enhanced Industry-Reported Comfort, Accessibility, and Safety Rating System for VR Applications" (authored by Sara Vlahović, Lea Skorin-Kapov, and Željka Car) at the 15th International Conference on Quality of Multimedia Experience (QoMEX 2023) held in Ghent, Belgium.
We started work on a new project as of December 1st 2022.: EPIIC: Enhanced Pilot Interfaces & Interactions for fighter Cockpit (Napredna pilotska sučelja i interakcije za kabinu borbenog zrakoplova), European defense fund 2021., in coopeartion with 25 European companies and universities and coordinated by Thales AVS, France SAS.
In June 2022, the 10th TMA PhD School on network intelligence and measurements was held at the University of Twente in the Dutch city of Enschede, where PhD students presented their research and attended lectures and lab sessions on related topics. Ivan Bartolec, a member of MUEXLab, attended the School and presented the poster "Performance Estimation of Encrypted Video Streaming Considering End-User Playback-Related Interactions".
We are happy to announce that our team's paper The Impact of Network and Social Context on Quality of Experience for Competitive Multiplayer Virtual Reality Games presented by Sara Vlahović at GameSys2022 in Athlone, Ireland was awarded the MMVE & GameSys Best Paper Award! Also check out our dataset paper entitled GCD: A Cloud Gaming Dataset with Gameplay Video and Network Recordings, presented by Ivan Slivar at MMSys2022, Open Dataset & Software Track. The dataset is available here.
In collaboration with the companies Inceptum, Storm Computers, and Databox, we arranged a visit to the Databox data center in Zagreb for students enrolled in the Network Performance and Traffic course. Students in their first year of Network Science profile had the opportunity to hear presentations about the companies Storm Computers and Databox and see for themselves how the actual data center is structured. Students had the opportunity to view the most recent equipment and hear Vladimir Olujić's practice-based insights. Following the visit, Davor Kobal of the company Inceptum presented the final lecture on OSS and BSS systems. Thank you to Storm Computers, Databox, and Inceptum. The students were very pleased with this method of lecture, and it is now up to us to provide more opportunities for learning and connecting in the future.
The Department of Telecommunications hosted open days on June 1st. At this event, the research laboratories of our department presented their research areas and the works of their former and current students to all interested students. Works relating to the development of networked games, virtual reality games, cloud gaming over mobile access networks, and augmented reality application development were displayed in our laboratory. Numerous students expressed interest in the topics covered by our laboratory, and our undergraduate, graduate, and doctoral students answered all questions.
A review paper published in Journal on Multimodal User Interfaces: “A survey of challenges and methods for Quality of Experience assessment of interactive VR applications”, authors Sara Vlahović, Mirko Sužnjević, and Lea Skorin-Kapov. The paper is available here.
Dataset paper accepted at MMSys 2022 “CGD: A Cloud Gaming Dataset with Gameplay Video and Network Recordings”, authors Ivan Slivar, Kresimir Bačić, Irena Oršolić, Lea Skorin-Kapov, Mirko Sužnjević. The dataset is available here.
Paper accepted to the 2nd Workshop on Game Systems (GameSys'22), collocated with MMSys 2022: “The Impact of Network and Social Context on Quality of Experience for Competitive Multiplayer Virtual Reality Games”, authors Ivan Slivar, Sara Vlahović, Matko Šilić, Lea Skorin-Kapov, Mirko Sužnjević.
The three-year long Edu4Games project was successfully completed on March 22, 2022. The final conference event was held at the Acadamy of Dramatic Art in Zagreb, and brought together project partners, students, industry representatives, representatives from the Ministry of Culture and Media, and representatives from the Croatian Audiovisual Centre.
MUEXlab members Sara Vlahović and Irena Oršolić organized a workshop on the topic of QoE for virtual reality (VR) applications on December 16 at FER. The workshop was a part of the Erasmus+ project VR Science Lab, with participants being high-school students from Croatia, the Czech Republic, Turkey, and Romania.
The workshop consisted of two lectures, group work with an elevator pitch, and a VR demo assisted by MUEXlab member Krešimir Bačić.
MUEXlab is looking for a new PhD student to join the team as of early 2022.
Are you interested in robot control and remote reality? How about developing innovative applications in game engines such as Unity? Would you like to enroll in the doctoral study program at FER?
We will be offering a 4-year long fully funded position. Please contact mirko.suznjevic@fer.hr to find out more.
Ivan Bartolec presented the paper entitled "Performance Estimation of Encrypted Video Streaming in Light of End-User Playback-Related Interactions" at the Doctoral Symposium of the Multimedia Systems Conference (MMSys 2021) in Istanbul, Turkey.
Congratulations to team member Ana Agić for successfully defending her PhD thesis at the Faculty of Graphic Arts entitled "Enhancing the Virtual Reality User Experience by Setting Scene Design Parameters and Locomotion Techniques", under the supervision of prof. Lidija Mandić and prof. Lea Skorin-Kapov. Great job Ana!
Congratulations to team member Dunja Vučić for successfully defending her PhD thesis entitled "Quality of Experience Driven Video Encoding Adaptation for Multiparty Audiovisual Telemeetings on Mobile Devices", under the supervision of prof. Lea Skorin-Kapov. Great job Dunja!
MUEXlab member Sara Vlahović won the Best PhD Forum presentation award at the 16th International Conference on Telecommunications (Zagreb, Croatia, June 30–July 2, 2021). In the Forum, she presented her PhD topic entitled "QoE Assessment for Virtual Reality Gaming".
Prof. Lea Skorin-Kapov gave a keynote talk at the 16th International Conference on Telecommunications (Zagreb, Croatia, June 30–July 2, 2021) entitled “Quality of Experience in the Era of Immersive Networked Multimedia Services”. Further details can be found here. The talk was also organized by the IEEE ComSoc Croatia Chapter.
The paper entitled "On Machine Learning Based Video QoE Estimation Across Different Networks", authored by MUEXlab member Irena Oršolić and Michael Seufert (University of Würzburg, Germany) won the Best Paper Award at the 16th International Conference on Telecommunications (Zagreb, Croatia, June 30–July 2, 2021).
MUEXlab members won the Best Paper Award at the 13th International Conference on Quality of Multimedia Experience (held online June 14–17, 2021) for the paper entitled "QoE Assessment of FPV Drone Control in a Cloud Gaming Based Simulation", authored by Matko Šilić, Mirko Sužnjević, and Lea Skorin-Kapov. Congratulations!
The following papers authored by MUEXlab members got accepted for publication at the 16th International Conference on Telecommunications (Zagreb, Croatia, June 30–July 2, 2021):
- "On Machine Learning Based Video QoE Estimation Across Different Networks", authors Irena Oršolić and Michael Seufert
- "Investigation of the Relationship Between Subjective and Objective Video Quality Metrics for Multiparty Video Calls on Mobile Devices", authors Dunja Vučić and Lea Skorin-Kapov
Our member Mirko Sužnjević gave an interview about the Edu4Games project aimed at creating standards of professions and qualifications as well as new study programs for the area of design and development of video games. The article is in Croatian language and available here.
The following papers authored by MUEXlab members got accepted at the 13th International Conference on Quality of Multimedia Experience (to be held online June 14–17, 2021):
- "The Effect of VR Gaming on Discomfort, Cybersickness, and Reaction Time", authors Sara Vlahović, Mirko Sužnjević, Nina Pavlin-Bernardić, and Lea Skorin-Kapov
- "QoE Assessment of FPV Drone Control in a Cloud Gaming Based Simulation", authors Matko Šilić, Mirko Sužnjević, and Lea Skorin-Kapov
VOLCO: Evaluation and Optimization of Volumetric Video QoE for Services Based on Holographic Communication of Remote Participants (Procjena i optimizacija iskustvene kvalitete volumetrijskog videa za usluge temeljene na hologramskoj komunikaciji udaljenih sudionika) has been succesfully completed. The project (KK.03.2.2.03.0095) was funded by the European Regional Development Fund (Koherentnost i kohezija 2014. - 2020.), in cooperation with the company Diversitas IT sustavi d.o.o. Under this project, the MUEXLab research team explored different solutions for volumetric capture and processing, and analyzed different metrics for the evaluation of volumetric video quality. The results of this project will contribute towards the development of a volumetric video communication and collaboration system.
New project started as of June 24th : VOLCO: Evaluation and Optimization of Volumetric Video QoE for Services Based on Holographic Communication of Remote Participants (Procjena i optimizacija iskustvene kvalitete volumentrijskog videa za usluge temeljene na hologramskoj komunikaciji udaljenih sudionika), Europski fond za regionalni razvoj, Konkurentnost i Kohezija 2014. - 2020., in cooperation with the company Diversitas IT sustavi d.o.o..
MUEXlab members Sara Vlahović and Ivan Bartolec attended the fourth annual FER Day at MIOC on 9th of March 2020. The event was intended for students of the XV. Gymnasium, Zagreb (popularly called MIOC) to get acquainted with the possibilities of studying at FER, scientific and professional projects, job opportunities, but also social activities, entertainment, sports, artistic, and cultural activites that FER offers to its students. Sara and Ivan demonstrated HTC Vive Virtual Reality System at a stand, where they introduced MUEXlab and its ongoing research activities regarding Virtual Reality. Sara also held a lecture entitled "From MIOC reality to Virtual Reality" where she talked about her academic and scientific experience, as a former student at MIOC and FER, and as a researcher today.
The Minister of Economy, Entrepreneurship and Crafts of the Republic of Croatia, Mr. Darko Horvat, visited the Center for Articial Intelligence at the Faculty of Electrical Engineering and Computing on October 14th, 2019. In the scope of the visit, the Minister and his collegues visited the Department of Telecommunications, where they met with reperesentatives from MUEXlab and IoTLab, both of which are a part of the Center for AI. MUEXlab representatives Lea Skorin-Kapov and Mirko Sužnjević discussed ongoing research activities and projects.
This years PhD Forum was held as part of the 27th International Conference on Software, Telecommunications and Computer Networks (SoftCOM 2019), in Split, Croatia, and was attended by 3 MUEXlab members. The event was organized with the help from Mirko Sužnjević who acted as a PhD Forum committee member, and a representative from FER. To participate on the PhD Forum, PhD students had to write a short abstract of their current research, hold a 2 minute pitch talk, and prepare and present a poster. Sara Vlahović presented her research area and results with the abstract and poster entitled QoE Assessment for Interactive AR/VR Applications, receiving the award for best pitch talk presentation. Ivan Bartolec presented his abstract and poster entitled The Implications of End-user Service Usage Behavior Patterns on In-network Video QoE Monitoring and Management, winning the second place for best pitch talk presentation.
EPSS19 Summer School on Machine Learning and Data Mining for Geo-Spatial Data, Quality of Experience and Human-Computer Interaction took place from 11th September until 16th September, in Würzburg, Germany, as part of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD). It was attended by three MUEXlab members, Irena Oršolić and Ivan Bartolec, whom were both part of the QoE track of the Summer School, while Prof. Lea Skorin-Kapov held a lecture entitled "Assessing and Modeling Gaming QoE". The summer school had a strong focus on practical aspects. The participants got to know and learned to apply various machine learning and data mining techniques in their field of research.
EPSS19: Link to Summer School web page
ECML: Link to Conference web page
Social media: You can find photos from the event here
Representatives from Rimac Automobili visted MUEXlab as a part of their visit to the Faculty of Electrical Engineering and Computing on Sept 6, 2019. Rimac Automobili is a high-tech Croatian-based company, manufacturing electric hypercars and providing full technology solutions to global automotive manufacturers. During the visit, we presented our ongoing research activities, with a focus on various applications of Virtual and Augmented Reality.
On 13th of June 2019, 17 undergraduate students visited Ericsson Nikola Tesla Zagreb as a part of the Multimedia Services course. The company visit provided students with the opportunity to learn more about the company, history of its collaboration with the Department of Telecommunications, new technology challenges and Ericsson's area of expertise. Students took a tour of the facility and got a first hand-look at network equipment and VR tile-based streaming service. Students were also given the opportunity to talk with people who are involved in the R&D. It was a great experience overall and students who were engaged the most won prizes. You can find the photo gallery here.
The following papers were presented at QoMEX 2019 in June in Berlin:
- "Fundamental Relationships for Deriving QoE in Systems", authors Tobias Hoßfeld, Poul E. Heegaard, Lea Skorin-Kapov, and Martin Varela
- "The Impact of Latency on Gaming QoE in Virtual Reality Games", authors Sara Vlahović, Mirko Sužnjević, Lea Skorin-Kapov
- "To Share or Not to Share? How Exploitation of Context Data Can Improve In-Network QoE Monitoring of Encrypted YouTube Streams", authors Irena Oršolić, Lea Skorin-Kapov, Tobias Hossfeld
- "In-Network YouTube Performance Estimation in Light of End User Playback-Related Interactions", authors Ivan Bartolec, Irena Oršolić, Lea Skorin-Kapov
Mirko Sužnjević held an invited talk at a developers conference CASE 2019. The talk was entitled: "Quality of Experience for Virtual Reality Technologies".
Mirko Suznjevic held and invited lecture on the Faculty of Teacher Education in scope of the "ICT in upbringing and education" course. The lecture was titled: "Virtual and augmented reality in early education".
Our PhD students, Irena Oršolić and Ivan Bartolec attended The 14th International Conference on Network and Service Management (CNSM 2018) in Rome, Italy. Irena presented a paper entitled In-Network QoE and KPI Monitoring of Mobile YouTube Traffic: Insights for Encrypted iOS Flows, co-authored with Petra Rebernjak, Mirko Sužnjević, and Lea Skorin-Kapov. She was also awarded with a student travel grant.
Members of the MUEXlab will present an application demonstrating different locomotion techniques in Virtual Reality at the University of Zagreb Fair. The fair will be held from November 21-24 this year in the premises of the Student Center in Zagreb. All interested students are invited to come visit us at our booth!
Our PhD students Sara Vlahović, Ivan Bartolec, and Irena Oršolić attended the PhD forum organized in the scope of International Conference on Smart Systems and Technologies (SST 2018), Osijek, Croatia. Irena Oršolić presented research results and challenges related to In-network Quality of Experience Monitoring of HTTP Adaptive Streaming Services, and received the second best pitch talk award.
New project started as of Sept. 1st : ARIEN: Power infrustructrure management using collaboration techniques in Augmented Reality (Upravljanje energetskom infrastrukturom kroz kolaboraciju u proširenoj stvarnosti), 2018-2020, Europski fond za regionalni razvoj, IRI project, in cooperation with the company Diversitas IT sustavi d.o.o..
Paper on In-Network QoE and KPI Monitoring of Mobile YouTube Traffic: Insights for Encrypted iOS Flows accepted for publication at CNSM 2018 (Rome).