Sparse pixel image sensor

TL;DR

This paper introduces a novel sparse pixel image sensor that captures only the most relevant image features, enabling efficient classification of images like handwritten digits with reduced delay and energy use.

Contribution

The paper presents a new image sensor design that uses a tunable array of photodetectors to classify images directly, bypassing full image reconstruction.

Findings

Achieved handwritten digit classification accuracy comparable to traditional methods.

Reduced energy consumption and latency compared to conventional cameras.

Demonstrated the effectiveness of sparse sampling in optical image classification.

Abstract

As conventional frame-based cameras suffer from high energy consumption and latency, several new types of image sensors have been devised, with some of them exploiting the sparsity of natural images in some transform domains. Instead of sampling the full image, those devices capture only the coefficients of the most relevant spatial frequencies. The number of samples can be even sparser if a signal only needs to be classified rather than being fully reconstructed. Based on this mathematical framework, we developed an image sensor that can be trained to classify optically projected images by reading out the few most relevant pixels. The device is based on a two-dimensional array of metal-semiconductor-metal photodetectors with individually tunable photoresponsivity values. We demonstrate its use for the classification of handwritten digits with an accuracy comparable to that of conventional systems, but with lower delay and energy consumption.