Constructing Per-Shot Bitrate Ladders using Visual Information Fidelity

TL;DR

This paper introduces a perceptually optimized method for constructing per-shot bitrate and quality ladders using Visual Information Fidelity, enabling efficient and high-quality adaptive streaming without extensive encoding.

Contribution

It develops a novel prediction model for per-shot bitrate and quality ladders based on VIF features, improving efficiency and performance over existing content-adaptive methods.

Findings

Outperforms fixed bitrate ladders in bitrate and quality

Achieves near-reference ladder performance with less computation

Provides significant computational advantages

Abstract

Video service providers need their delivery systems to be able to adapt to network conditions, user preferences, display settings, and other factors. HTTP Adaptive Streaming (HAS) offers dynamic switching between different video representations to simultaneously enhance bandwidth consumption and users' streaming experiences. Per-shot encoding, pioneered by Netflix, optimizes the encoding parameters on each scene or shot. The Dynamic Optimizer (DO) uses the Video Multi-Method Assessment Fusion (VMAF) perceptual video quality prediction engine to deliver high-quality videos at reduced bitrates. Here we develop a perceptually optimized method of constructing optimal per-shot bitrate and quality ladders, using an ensemble of low-level features and Visual Information Fidelity (VIF) features. During inference, our method predicts the bitrate or quality ladder of a source video without any compression or quality estimation. We compare the performance of our model against other content-adaptive bitrate ladder prediction methods, a fixed bitrate ladder, and reference bitrate ladders constructed via exhaustive encoding using Bjontegaard-delta (BD) metrics. Our proposed method shows excellent gains in bitrate and quality against the fixed bitrate ladder and only small losses against the reference bitrate ladder, while providing significant computational advantages.