Which Adaptation Logic? An Objective and Subjective Performance Evaluation of HTTP-based Adaptive Media Streaming Systems

TL;DR

This paper comprehensively evaluates ten HTTP-based adaptive streaming algorithms through objective tests and real-world crowdsourcing, providing insights for deployment and a methodology for future comparisons.

Contribution

It offers the first extensive comparative analysis of multiple adaptation logics for DASH, including both controlled and real-world evaluations.

Findings

Different adaptation logics vary significantly in performance.

The evaluation methodology can be used for future algorithm assessments.

Results inform optimal strategies for real-world deployment.

Abstract

Multimedia content delivery over the Internet is predominantly using the Hypertext Transfer Protocol (HTTP) as its primary protocol and multiple proprietary solutions exits. The MPEG standard Dynamic Adaptive Streaming over HTTP (DASH) provides an interoperable solution and in recent years various adaptation logics/algorithms have been proposed. However, to the best of our knowledge, there is no comprehensive evaluation of the various logics/algorithms. Therefore, this paper provides a comprehensive evaluation of ten different adaptation logics/algorithms, which have been proposed in the past years. The evaluation is done both objectively and subjectively. The former is using a predefined bandwidth trajectory within a controlled environment and the latter is done in a real-world environment adopting crowdsourcing. The results shall provide insights about which strategy can be adopted in actual deployment scenarios. Additionally, the evaluation methodology described in this paper can be used to evaluate any other/new adaptation logic and to compare it directly with the results reported here.