Multi-Task Learning for Screen Content Image Coding

TL;DR

This paper introduces a learning-based image coding model specifically designed for screen content images that contain both synthetic and natural regions, enhancing compression efficiency by jointly learning reconstruction and segmentation.

Contribution

The paper presents a novel multi-task learning approach that produces a segmentation-friendly latent representation for improved screen content image coding, applicable even without the segmentation task during inference.

Findings

The proposed codec outperforms traditional methods on mixed-content SCIs.

Joint training for reconstruction and segmentation improves compression quality.

Segmentation information enhances the latent representation for better coding efficiency.

Abstract

With the rise of remote work and collaboration, compression of screen content images (SCI) is becoming increasingly important. While there are efficient codecs for natural images, as well as codecs for purely-synthetic images, those SCIs that contain both synthetic and natural content pose a particular challenge. In this paper, we propose a learning-based image coding model developed for such SCIs. By training an encoder to provide a latent representation suitable for two tasks -- input reconstruction and synthetic/natural region segmentation -- we create an effective SCI image codec whose strong performance is verified through experiments. Once trained, the second task (segmentation) need not be used; the codec still benefits from the segmentation-friendly latent representation.