Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations

TL;DR

This paper introduces Maximum Manifold Capacity Representations (MMCR), a novel method for optimizing the manifold capacity in neural representations, improving efficiency and class separability in natural image coding.

Contribution

The paper presents a new approach to directly optimize manifold capacity, enabling practical use of this efficiency metric in neural coding models.

Findings

MMCR achieves competitive results on SSL benchmarks.

Differences between MMCR and other SSL methods reveal insights into class separability.

MMCR models show high neural predictivity for the ventral stream.

Abstract

The efficient coding hypothesis proposes that the response properties of sensory systems are adapted to the statistics of their inputs such that they capture maximal information about the environment, subject to biological constraints. While elegant, information theoretic properties are notoriously difficult to measure in practical settings or to employ as objective functions in optimization. This difficulty has necessitated that computational models designed to test the hypothesis employ several different information metrics ranging from approximations and lower bounds to proxy measures like reconstruction error. Recent theoretical advances have characterized a novel and ecologically relevant efficiency metric, the manifold capacity, which is the number of object categories that may be represented in a linearly separable fashion. However, calculating manifold capacity is a computationally intensive iterative procedure that until now has precluded its use as an objective. Here we outline the simplifying assumptions that allow manifold capacity to be optimized directly, yielding Maximum Manifold Capacity Representations (MMCR). The resulting method is closely related to and inspired by advances in the field of self supervised learning (SSL), and we demonstrate that MMCRs are competitive with state of the art results on standard SSL benchmarks. Empirical analyses reveal differences between MMCRs and representations learned by other SSL frameworks, and suggest a mechanism by which manifold compression gives rise to class separability. Finally we evaluate a set of SSL methods on a suite of neural predictivity benchmarks, and find MMCRs are higly competitive as models of the ventral stream.