Do Neural Networks Compress Manifolds Optimally?

TL;DR

This paper investigates the optimal compression limits for low-dimensional manifolds with circular structures and finds that current neural network compressors do not achieve these theoretical optima.

Contribution

It provides the first analysis of optimal entropy-distortion tradeoffs for specific low-dimensional manifolds, revealing limitations of existing neural network compressors.

Findings

Optimal entropy-distortion bounds are derived for circular manifolds.

State-of-the-art neural network compressors do not reach these bounds.

The results highlight the need for improved compression methods for structured manifolds.

Abstract

Artificial Neural-Network-based (ANN-based) lossy compressors have recently obtained striking results on several sources. Their success may be ascribed to an ability to identify the structure of low-dimensional manifolds in high-dimensional ambient spaces. Indeed, prior work has shown that ANN-based compressors can achieve the optimal entropy-distortion curve for some such sources. In contrast, we determine the optimal entropy-distortion tradeoffs for two low-dimensional manifolds with circular structure and show that state-of-the-art ANN-based compressors fail to optimally compress them.