Semantic Segmentation in Learned Compressed Domain

TL;DR

This paper introduces a novel compressed domain approach for semantic segmentation that reduces redundancy and transforms compressed representations, achieving significant bitrate savings and faster inference compared to pixel domain methods.

Contribution

It proposes dynamic and static channel selection methods and transform modules to enhance segmentation in the compressed domain, outperforming existing pixel domain techniques.

Findings

Up to 15.8% bitrate savings over state-of-the-art compressed domain methods.

Approximately 83.6% bitrate reduction compared to pixel domain methods.

44.8% faster inference time in the compressed domain.

Abstract

Most machine vision tasks (e.g., semantic segmentation) are based on images encoded and decoded by image compression algorithms (e.g., JPEG). However, these decoded images in the pixel domain introduce distortion, and they are optimized for human perception, making the performance of machine vision tasks suboptimal. In this paper, we propose a method based on the compressed domain to improve segmentation tasks. i) A dynamic and a static channel selection method are proposed to reduce the redundancy of compressed representations that are obtained by encoding. ii) Two different transform modules are explored and analyzed to help the compressed representation be transformed as the features in the segmentation network. The experimental results show that we can save up to 15.8\% bitrates compared with a state-of-the-art compressed domain-based work while saving up to about 83.6\% bitrates and 44.8\% inference time compared with the pixel domain-based method.