Texture-guided Coding for Deep Features

TL;DR

This paper introduces a texture-guided feature compression method for deep neural network features, reducing data size while maintaining quality for machine tasks and enabling human interpretation through image reconstruction.

Contribution

It proposes a novel compression strategy that leverages textures to selectively compress features and reconstruct preview images, enhancing both machine efficiency and human understanding.

Findings

Effective feature data reduction with high-quality reconstruction.

Improved performance in feature compression tasks.

Supports human-machine interaction through preview images.

Abstract

With the rapid development of machine vision technology in recent years, many researchers have begun to focus on feature compression that is better suited for machine vision tasks. The target of feature compression is deep features, which arise from convolution in the middle layer of a pre-trained convolutional neural network. However, due to the large volume of data and high level of abstraction of deep features, their application is primarily limited to machine-centric scenarios, which poses significant constraints in situations requiring human-computer interaction. This paper investigates features and textures and proposes a texture-guided feature compression strategy based on their characteristics. Specifically, the strategy comprises feature layers and texture layers. The feature layers serve the machine, including a feature selection module and a feature reconstruction network. With the assistance of texture images, they selectively compress and transmit channels relevant to visual tasks, reducing feature data while providing high-quality features for the machine. The texture layers primarily serve humans and consist of an image reconstruction network. This image reconstruction network leverages features and texture images to reconstruct preview images for humans. Our method fully exploits the characteristics of texture and features. It eliminates feature redundancy, reconstructs high-quality preview images for humans, and supports decision-making. The experimental results demonstrate excellent performance when employing our proposed method to compress the deep features.