Estimation of optimal encoding ladders for tiled 360{\deg} VR video in adaptive streaming systems

TL;DR

This paper introduces a content-aware method for estimating optimal encoding ladders for tiled 360-degree VR videos in adaptive streaming, improving quality and resource efficiency over existing standards.

Contribution

It proposes a novel encoding ladder estimation approach using content features, distortion and resource models, and an ILP algorithm considering practical constraints.

Findings

Outperforms professional service provider encoding ladders in objective quality.

Provides cost-effective encoding solutions tailored to VR content.

Enhances immersive streaming experiences with optimized resource use.

Abstract

Given the significant industrial growth of demand for virtual reality (VR), 360{\deg} video streaming is one of the most important VR applications that require cost-optimal solutions to achieve widespread proliferation of VR technology. Because of its inherent variability of data-intensive content types and its tiled-based encoding and streaming, 360{\deg} video requires new encoding ladders in adaptive streaming systems to achieve cost-optimal and immersive streaming experiences. In this context, this paper targets both the provider's and client's perspectives and introduces a new content-aware encoding ladder estimation method for tiled 360{\deg} VR video in adaptive streaming systems. The proposed method first categories a given 360{\deg} video using its features of encoding complexity and estimates the visual distortion and resource cost of each bitrate level based on the proposed distortion and resource cost models. An optimal encoding ladder is then formed using the proposed integer linear programming (ILP) algorithm by considering practical constraints. Experimental results of the proposed method are compared with the recommended encoding ladders of professional streaming service providers. Evaluations show that the proposed encoding ladders deliver better results compared to the recommended encoding ladders in terms of objective quality for 360{\deg} video, providing optimal encoding ladders using a set of service provider's constraint parameters.