Advantages of Neural Population Coding for Deep Learning

TL;DR

This paper explores how using population coding in neural network output layers enhances robustness to noise and ambiguity, leading to improved accuracy in tasks like 3D object orientation prediction.

Contribution

It demonstrates both theoretically and empirically that population codes outperform single-neuron outputs and one-hot vectors in noisy and ambiguous scenarios.

Findings

Population codes improve robustness to input noise.

Population codes enhance accuracy in ambiguous output encoding.

Population codes outperform traditional methods in 3D pose prediction.

Abstract

Scalar variables, e.g., the orientation of a shape in an image, are commonly predicted using a single output neuron in a neural network. In contrast, the mammalian cortex represents variables with a population of neurons. In this population code, each neuron is most active at its preferred value and shows partial activity for other values. Here, we investigate the benefit of using a population code for the output layer of a neural network. We compare population codes against single-neuron outputs and one-hot vectors. First, we show theoretically and in experiments with synthetic data that population codes improve robustness to input noise in networks of stacked linear layers. Second, we demonstrate the benefit of using population codes to encode ambiguous outputs, such as the pose of symmetric objects. Using the T-LESS dataset of feature-less real-world objects, we show that population codes improve the accuracy of predicting 3D object orientation from image input.