Filling in the details: Perceiving from low fidelity images

TL;DR

This paper investigates how autoencoders can learn to reconstruct detailed images from low-fidelity, distorted inputs, revealing insights into the human-like perception capabilities of neural networks.

Contribution

It demonstrates that autoencoders can effectively learn to generate full-detail images from low-fidelity inputs, adapting to various distortions and mimicking human peripheral perception.

Findings

Autoencoders learn global features to compensate for low detail.

Networks accurately perceive color in the periphery despite achromatic input.

Autoencoders can reconstruct detailed images from 75% achromatic, low-fidelity inputs.

Abstract

Humans perceive their surroundings in great detail even though most of our visual field is reduced to low-fidelity color-deprived (e.g. dichromatic) input by the retina. In contrast, most deep learning architectures are computationally wasteful in that they consider every part of the input when performing an image processing task. Yet, the human visual system is able to perform visual reasoning despite having only a small fovea of high visual acuity. With this in mind, we wish to understand the extent to which connectionist architectures are able to learn from and reason with low acuity, distorted inputs. Specifically, we train autoencoders to generate full-detail images from low-detail "foveations" of those images and then measure their ability to reconstruct the full-detail images from the foveated versions. By varying the type of foveation, we can study how well the architectures can cope with various types of distortion. We find that the autoencoder compensates for lower detail by learning increasingly global feature functions. In many cases, the learnt features are suitable for reconstructing the original full-detail image. For example, we find that the networks accurately perceive color in the periphery, even when 75\% of the input is achromatic.