Extending Neural P-frame Codecs for B-frame Coding

TL;DR

This paper introduces a method to extend neural P-frame video codecs to efficiently encode B-frames by interpolating reference frames, resulting in significant bit-rate savings without quality loss.

Contribution

It proposes a simple extension to existing neural P-frame codecs to support B-frame coding through frame interpolation, enhancing compression efficiency.

Findings

Achieves 28.5% bit-rate reduction on UVG dataset

Uses frame interpolation to improve reference quality for B-frames

Easily integrates with existing neural P-frame codecs

Abstract

While most neural video codecs address P-frame coding (predicting each frame from past ones), in this paper we address B-frame compression (predicting frames using both past and future reference frames). Our B-frame solution is based on the existing P-frame methods. As a result, B-frame coding capability can easily be added to an existing neural codec. The basic idea of our B-frame coding method is to interpolate the two reference frames to generate a single reference frame and then use it together with an existing P-frame codec to encode the input B-frame. Our studies show that the interpolated frame is a much better reference for the P-frame codec compared to using the previous frame as is usually done. Our results show that using the proposed method with an existing P-frame codec can lead to 28.5%saving in bit-rate on the UVG dataset compared to the P-frame codec while generating the same video quality.