Revisit Visual Representation in Analytics Taxonomy: A Compression Perspective

TL;DR

This paper introduces a novel compression framework for visual data that supports multiple machine vision tasks at low bit-rates by leveraging transferability among tasks and a codebook-based hyperprior for efficient representation.

Contribution

It proposes a new approach to compress visual representations supporting diverse analytics tasks, utilizing a codebook-based hyperprior to improve compression efficiency and task support.

Findings

Supports multiple vision tasks at lower bit-rates

Outperforms existing compression schemes in experiments

Utilizes a codebook-based hyperprior for better entropy estimation

Abstract

Visual analytics have played an increasingly critical role in the Internet of Things, where massive visual signals have to be compressed and fed into machines. But facing such big data and constrained bandwidth capacity, existing image/video compression methods lead to very low-quality representations, while existing feature compression techniques fail to support diversified visual analytics applications/tasks with low-bit-rate representations. In this paper, we raise and study the novel problem of supporting multiple machine vision analytics tasks with the compressed visual representation, namely, the information compression problem in analytics taxonomy. By utilizing the intrinsic transferability among different tasks, our framework successfully constructs compact and expressive representations at low bit-rates to support a diversified set of machine vision tasks, including both high-level semantic-related tasks and mid-level geometry analytic tasks. In order to impose compactness in the representations, we propose a codebook-based hyperprior, which helps map the representation into a low-dimensional manifold. As it well fits the signal structure of the deep visual feature, it facilitates more accurate entropy estimation, and results in higher compression efficiency. With the proposed framework and the codebook-based hyperprior, we further investigate the relationship of different task features owning different levels of abstraction granularity. Experimental results demonstrate that with the proposed scheme, a set of diversified tasks can be supported at a significantly lower bit-rate, compared with existing compression schemes.