Revisiting the Sample Adaptive Offset post-filter of VVC with Neural-Networks

TL;DR

This paper enhances the Sample Adaptive Offset (SAO) filter in VVC using neural networks to improve coding efficiency, achieving at least 2.3% BD-rate gain with manageable complexity.

Contribution

It introduces a neural network-based approach to replace the a-priori classification in SAO, improving VVC performance over the standard SAO.

Findings

Achieves at least 2.3% BD-rate gain in Random Access mode.

Maintains relatively low complexity compared to other neural network methods.

Revisits and improves the SAO filter in VVC with neural networks.

Abstract

The Sample Adaptive Offset (SAO) filter has been introduced in HEVC to reduce general coding and banding artefacts in the reconstructed pictures, in complement to the De-Blocking Filter (DBF) which reduces artifacts at block boundaries specifically. The new video compression standard Versatile Video Coding (VVC) reduces the BD-rate by about 36% at the same reconstruction quality compared to HEVC. It implements an additional new in-loop Adaptive Loop Filter (ALF) on top of the DBF and the SAO filter, the latter remaining unchanged compared to HEVC. However, the relative performance of SAO in VVC has been lowered significantly. In this paper, it is proposed to revisit the SAO filter using Neural Networks (NN). The general principles of the SAO are kept, but the a-priori classification of SAO is replaced with a set of neural networks that determine which reconstructed samples should be corrected and in which proportion. Similarly to the original SAO, some parameters are determined at the encoder side and encoded per CTU. The average BD-rate gain of the proposed SAO improves VVC by at least 2.3% in Random Access while the overall complexity is kept relatively small compared to other NN-based methods.