A photosensor employing data-driven binning for ultrafast image recognition

TL;DR

This paper introduces a data-driven binning approach for ultrafast image recognition, combining sensor elements into a superpixel optimized via machine learning, enabling nanosecond classification without accuracy loss.

Contribution

It presents a novel sensor design that uses machine learning to optimize pixel binning for ultrafast recognition, surpassing traditional methods.

Findings

Achieved nanosecond image classification using a single superpixel.

Enhanced sensitivity without sacrificing classification accuracy.

Applicable to various optical sensing applications.

Abstract

Pixel binning is a technique, widely used in optical image acquisition and spectroscopy, in which adjacent detector elements of an image sensor are combined into larger pixels. This reduces the amount of data to be processed as well as the impact of noise, but comes at the cost of a loss of information. Here, we push the concept of binning to its limit by combining a large fraction of the sensor elements into a single superpixel that extends over the whole face of the chip. For a given pattern recognition task, its optimal shape is determined from training data using a machine learning algorithm. We demonstrate the classification of optically projected images from the MNIST dataset on a nanosecond timescale, with enhanced sensitivity and without loss of classification accuracy. Our concept is not limited to imaging alone but can also be applied in optical spectroscopy or other sensing applications.