Minimalist Vision with Freeform Pixels

TL;DR

This paper introduces minimalist vision systems with freeform pixels, designed as neural network layers, achieving comparable performance to traditional cameras with fewer pixels, enhancing privacy and enabling self-powered operation.

Contribution

The paper presents a novel approach to designing minimalist cameras with freeform pixels optimized via neural network training, demonstrating effective vision tasks with significantly fewer pixels.

Findings

Achieved comparable performance to traditional cameras with only 8 pixels.

Designed minimalist cameras for indoor monitoring, lighting measurement, and traffic estimation.

Showed that minimalist cameras can be self-powered due to low measurement requirements.

Abstract

A minimalist vision system uses the smallest number of pixels needed to solve a vision task. While traditional cameras use a large grid of square pixels, a minimalist camera uses freeform pixels that can take on arbitrary shapes to increase their information content. We show that the hardware of a minimalist camera can be modeled as the first layer of a neural network, where the subsequent layers are used for inference. Training the network for any given task yields the shapes of the camera's freeform pixels, each of which is implemented using a photodetector and an optical mask. We have designed minimalist cameras for monitoring indoor spaces (with 8 pixels), measuring room lighting (with 8 pixels), and estimating traffic flow (with 8 pixels). The performance demonstrated by these systems is on par with a traditional camera with orders of magnitude more pixels. Minimalist vision has two major advantages. First, it naturally tends to preserve the privacy of individuals in the scene since the captured information is inadequate for extracting visual details. Second, since the number of measurements made by a minimalist camera is very small, we show that it can be fully self-powered, i.e., function without an external power supply or a battery.