Visual Analysis Motivated Rate-Distortion Model for Image Coding

TL;DR

This paper introduces a novel rate-distortion model for image coding that enhances visual analysis tasks by optimizing bit allocation and distortion measurement, leading to significant bitrate savings while maintaining analysis performance.

Contribution

It presents a new rate allocation strategy and a visual analysis-friendly distortion model tailored for VVC intra compression, improving analysis accuracy at lower bitrates.

Findings

Achieves up to 28.17% bitrate reduction

Maintains analysis performance across tasks

Introduces multi-scale feature distortion metric

Abstract

Optimized for pixel fidelity metrics, images compressed by existing image codec are facing systematic challenges when used for visual analysis tasks, especially under low-bitrate coding. This paper proposes a visual analysis-motivated rate-distortion model for Versatile Video Coding (VVC) intra compression. The proposed model has two major contributions, a novel rate allocation strategy and a new distortion measurement model. We first propose the region of interest for machine (ROIM) to evaluate the degree of importance for each coding tree unit (CTU) in visual analysis. Then, a novel CTU-level bit allocation model is proposed based on ROIM and the local texture characteristics of each CTU. After an in-depth analysis of multiple distortion models, a visual analysis friendly distortion criteria is subsequently proposed by extracting deep feature of each coding unit (CU). To alleviate the problem of lacking spatial context information when calculating the distortion of each CU, we finally propose a multi-scale feature distortion (MSFD) metric using different neighboring pixels by weighting the extracted deep features in each scale. Extensive experimental results show that the proposed scheme could achieve up to 28.17\% bitrate saving under the same analysis performance among several typical visual analysis tasks such as image classification, object detection, and semantic segmentation.