Learned Video Codec with Enriched Reconstruction for CLIC P-frame Coding

TL;DR

This paper introduces a novel learning-based video codec for P-frame coding, utilizing specialized networks for residuals and motion estimation, achieving competitive quality metrics in the CLIC challenge.

Contribution

It presents a new codec architecture with Refine-Net and hierarchical ME-Net, enhancing residual coding and motion estimation for improved video compression.

Findings

Achieved performance comparable to top CLIC challenge entrants

Demonstrated effectiveness of attention-based motion estimation

Validated through extensive ablation studies

Abstract

This paper proposes a learning-based video codec, specifically used for Challenge on Learned Image Compression (CLIC, CVPRWorkshop) 2020 P-frame coding. More specifically, we designed a compressor network with Refine-Net for coding residual signals and motion vectors. Also, for motion estimation, we introduced a hierarchical, attention-based ME-Net. To verify our design, we conducted an extensive ablation study on our modules and different input formats. Our video codec demonstrates its performance by using the perfect reference frame at the decoder side specified by the CLIC P-frame Challenge. The experimental result shows that our proposed codec is very competitive with the Challenge top performers in terms of quality metrics.