Entropy Transformer Networks: A Learning Approach via Tangent Bundle Data Manifold

TL;DR

This paper introduces Entropy Transformer Networks (ESTN), a novel interpolation method on data manifolds that enhances CNN performance under scale variations by preserving gradient norms and improving accuracy.

Contribution

The paper proposes ESTN, which interpolates on data manifolds using tangent spaces and entropy regularization, addressing limitations of standard spatial transformer networks.

Findings

Improves accuracy on image reconstruction and classification tasks.

Reduces computational cost compared to existing methods.

Enhances gradient norm preservation during training.

Abstract

This paper focuses on an accurate and fast interpolation approach for image transformation employed in the design of CNN architectures. Standard Spatial Transformer Networks (STNs) use bilinear or linear interpolation as their interpolation, with unrealistic assumptions about the underlying data distributions, which leads to poor performance under scale variations. Moreover, STNs do not preserve the norm of gradients in propagation due to their dependency on sparse neighboring pixels. To address this problem, a novel Entropy STN (ESTN) is proposed that interpolates on the data manifold distributions. In particular, random samples are generated for each pixel in association with the tangent space of the data manifold and construct a linear approximation of their intensity values with an entropy regularizer to compute the transformer parameters. A simple yet effective technique is also proposed to normalize the non-zero values of the convolution operation, to fine-tune the layers for gradients' norm-regularization during training. Experiments on challenging benchmarks show that the proposed ESTN can improve predictive accuracy over a range of computer vision tasks, including image reconstruction, and classification, while reducing the computational cost.