Motion-Plane-Adaptive Inter Prediction in 360-Degree Video Coding

TL;DR

This paper introduces a novel motion-plane-adaptive inter prediction method for 360-degree video that accounts for spherical characteristics, improving compression efficiency over traditional 2D mapping approaches.

Contribution

It proposes a new inter prediction technique and motion vector prediction method that operate on 3D motion planes, enhancing 360-degree video coding performance.

Findings

Achieves up to 3.97% Bjontegaard Delta rate savings on PSNR.

Achieves up to 3.40% Bjontegaard Delta rate savings on WS-PSNR.

Significant improvement over baseline VVC standard.

Abstract

Inter prediction is one of the key technologies enabling the high compression efficiency of modern video coding standards. 360-degree video needs to be mapped to the 2D image plane prior to coding in order to allow compression using existing video coding standards. The distortions that inevitably occur when mapping spherical data onto the 2D image plane, however, impair the performance of classical inter prediction techniques. In this paper, we propose a motion-plane-adaptive inter prediction technique (MPA) for 360-degree video that takes the spherical characteristics of 360-degree video into account. Based on the known projection format of the video, MPA allows to perform inter prediction on different motion planes in 3D space instead of having to work on the - in theory arbitrarily mapped - 2D image representation directly. We furthermore derive a motion-plane-adaptive motion vector prediction technique (MPA-MVP) that allows to translate motion information between different motion planes and motion models. Our proposed integration of MPA together with MPA-MVP into the state-of-the-art H.266/VVC video coding standard shows significant Bjontegaard Delta rate savings of 1.72% with a peak of 3.97% based on PSNR and 1.56% with a peak of 3.40% based on WS-PSNR compared to the VTM-14.2 baseline on average.