Learning in situ: a randomized experiment in video streaming
Francis Y. Yan, Hudson Ayers, Chenzhi Zhu, Sadjad Fouladi, James Hong,, Keyi Zhang, Philip Levis, Keith Winstein

TL;DR
This study conducts a large-scale randomized trial of video streaming algorithms, revealing challenges for machine learning approaches in real-world conditions and proposing a hybrid control method that outperforms existing schemes.
Contribution
It provides empirical evidence on the performance of streaming algorithms in real-world settings and introduces a robust hybrid ABR algorithm combining classical control with learned network prediction.
Findings
Simple buffer-based control often outperforms complex algorithms in practice.
Heavy-tailed network behavior complicates the deployment of learned algorithms.
The proposed hybrid algorithm outperforms other schemes in real-world tests.
Abstract
We describe the results of a randomized controlled trial of video-streaming algorithms for bitrate selection and network prediction. Over the last eight months, we have streamed 14.2 years of video to 56,000 users across the Internet. Sessions are randomized in blinded fashion among algorithms, and client telemetry is recorded for analysis. We found that in this real-world setting, it is difficult for sophisticated or machine-learned control schemes to outperform a "simple" scheme (buffer-based control), notwithstanding good performance in network emulators or simulators. We performed a statistical analysis and found that the variability and heavy-tailed nature of network and algorithm behavior create hurdles for robust learned algorithms in this area. We developed an ABR algorithm that robustly outperforms other schemes in practice, by combining classical control with a learned…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImage and Video Quality Assessment · Network Traffic and Congestion Control · Internet Traffic Analysis and Secure E-voting
