Efficient coding along the visual hierarchy

TL;DR

This paper demonstrates that an unsupervised efficient coding approach can develop human-aligned visual features from limited data, aligning with neural responses and improving low-data learning.

Contribution

It introduces a hierarchical unsupervised learning method based on efficient coding that captures natural image statistics without labels or backpropagation.

Findings

Features progress from edges to textures and shapes.

Features are recognizable by humans and predict fMRI responses.

Hybrid training improves brain alignment and category learning in low-data scenarios.

Abstract

Biological visual systems learn from limited experience, unlike deep learning models that rely on millions of training images. What learning principles make this possible? We tested whether efficient coding, the idea that neural representations capture the statistical structure of natural inputs, can build a hierarchy of human-aligned visual features from limited data. We developed an unsupervised learning procedure in which each layer of a deep network compresses its inputs onto the dominant modes of variation in natural images, using only local statistics and no labels, tasks, or backpropagation. This unsupervised procedure yields features that progress from edges and colors to textures and shapes. The features of this deep efficient coding model are readily recognized by human observers and are predictive of image-evoked fMRI responses in human visual cortex. Furthermore, a hybrid learning procedure that combines efficient coding with supervised fine-tuning yields better brain alignment in low-data settings and more rapid category learning. These findings suggest that efficient coding may shape representations across the entire visual hierarchy and help explain the data efficiency of biological vision.