A Survey on Perceptually Optimized Video Coding

TL;DR

This survey comprehensively reviews perceptually optimized video coding, covering models, methods, and standards to enhance compression efficiency by exploiting visual redundancies for high-quality video delivery.

Contribution

It systematically summarizes recent advances, frameworks, and challenges in perceptually optimized video coding, providing a valuable reference for future research and development.

Findings

Perceptual models improve coding efficiency.

Recent standards incorporate perceptual optimization techniques.

Challenges include balancing quality and computational complexity.

Abstract

To provide users with more realistic visual experiences, videos are developing in the trends of Ultra High Definition (UHD), High Frame Rate (HFR), High Dynamic Range (HDR), Wide Color Gammut (WCG) and high clarity. However, the data amount of videos increases exponentially, which requires high efficiency video compression for storage and network transmission. Perceptually optimized video coding aims to maximize compression efficiency by exploiting visual redundancies. In this paper, we present a broad and systematic survey on perceptually optimized video coding. Firstly, we present problem formulation and framework of the perceptually optimized video coding, which includes visual perception modelling, visual quality assessment and perceptual video coding optimization. Secondly, recent advances on visual factors, computational perceptual models and quality assessment models are presented. Thirdly, we review perceptual video coding optimizations from four key aspects, including perceptually optimized bit allocation, rate-distortion optimization, transform and quantization, filtering and enhancement. In each part, problem formulation, working flow, recent advances, advantages and challenges are presented. Fourthly, perceptual coding performances of the latest coding standards and tools are experimentally analyzed. Finally, challenging issues and future opportunities are identified.