Object-Based Image Coding: A Learning-Driven Revisit

TL;DR

This paper revisits object-based image coding by introducing a learning-driven approach that employs neural networks for efficient object segmentation and separate processing of foreground and background, achieving superior quality at very low bitrates.

Contribution

It proposes an end-to-end learning framework for object-based image coding that improves compact representation and visual quality at ultra-low bitrates, addressing a longstanding challenge.

Findings

Noticeable subjective quality improvement over JPEG2K, HEVC-based BPG, and other learned methods.

Effective processing of arbitrary-shaped objects through element-wise masking and neural compression.

Demonstrated performance on PASCAL VOC dataset at bitrate $ extless$ 0.1 bpp.

Abstract

The Object-Based Image Coding (OBIC) that was extensively studied about two decades ago, promised a vast application perspective for both ultra-low bitrate communication and high-level semantical content understanding, but it had rarely been used due to the inefficient compact representation of object with arbitrary shape. A fundamental issue behind is how to efficiently process the arbitrary-shaped objects at a fine granularity (e.g., feature element or pixel wise). To attack this, we have proposed to apply the element-wise masking and compression by devising an object segmentation network for image layer decomposition, and parallel convolution-based neural image compression networks to process masked foreground objects and background scene separately. All components are optimized in an end-to-end learning framework to intelligently weigh their (e.g., object and background) contributions for visually pleasant reconstruction. We have conducted comprehensive experiments to evaluate the performance on PASCAL VOC dataset at a very low bitrate scenario (e.g., $\lesssim$0.1 bits per pixel - bpp) which have demonstrated noticeable subjective quality improvement compared with JPEG2K, HEVC-based BPG and another learned image compression method. All relevant materials are made publicly accessible at https://njuvision.github.io/Neural-Object-Coding/.