Comparison of Popular Video Conferencing Apps Using Client-side Measurements on Different Backhaul Networks

TL;DR

This study compares Google Meet, Microsoft Teams, and Zoom in terms of performance, resource consumption, and user experience across different networks, providing insights for users and developers to optimize platform choice and design.

Contribution

The paper offers a comprehensive set of network and system measurements, along with user studies, to evaluate and compare popular video conferencing apps' performance and resource usage.

Findings

Significant differences in how apps handle video and audio streams.

Resource consumption varies notably among platforms.

No single app is optimal for all user needs.

Abstract

Video conferencing platforms have been appropriated during the COVID-19 pandemic for different purposes, including classroom teaching. However, the platforms are not designed for many of these objectives. When users, like educationists, select a platform, it is unclear which platform will perform better given the same network and hardware resources to meet the required Quality of Experience (QoE). Similarly, when developers design a new video conferencing platform, they do not have clear guidelines for making design choices given the QoE requirements. In this paper, we provide a set of networks and systems measurements, and quantitative user studies to measure the performance of video conferencing apps in terms of both, Quality of Service (QoS) and QoE. Using those metrics, we measure the performance of Google Meet, Microsoft Teams, and Zoom, which are three popular platforms in education and business. We find a substantial difference in how the three apps treat video and audio streams. We see that their choice of treatment affects their consumption of hardware resources. Our quantitative user studies confirm the findings of our quantitative measurements. While each platform has its benefits, we find that no app is ideal. A user can choose a suitable platform depending on which of the following, audio, video, or network bandwidth, CPU, or memory are more important.