Solving Vision Tasks with Simple Photoreceptors Instead of Cameras

TL;DR

This paper investigates the effectiveness of extremely simple photoreceptor-based sensors in solving vision tasks, demonstrating that minimal sensors can perform comparably to cameras when properly designed, with a focus on computational optimization.

Contribution

It introduces a computational design optimization method for simple visual sensors and compares their performance to manually designed sensors across various tasks.

Findings

Few photoreceptors can solve navigation and control tasks effectively.

Sensor design significantly impacts task performance.

Optimized sensor designs outperform manually devised designs in most cases.

Abstract

A de facto standard in solving computer vision problems is to use a common high-resolution camera and choose its placement on an agent (i.e., position and orientation) based on human intuition. On the other hand, extremely simple and well-designed visual sensors found throughout nature allow many organisms to perform diverse, complex behaviors. In this work, motivated by these examples, we raise the following questions: 1. How effective simple visual sensors are in solving vision tasks? 2. What role does their design play in their effectiveness? We explore simple sensors with resolutions as low as one-by-one pixel, representing a single photoreceptor First, we demonstrate that just a few photoreceptors can be enough to solve many tasks, such as visual navigation and continuous control, reasonably well, with performance comparable to that of a high-resolution camera. Second, we show that the design of these simple visual sensors plays a crucial role in their ability to provide useful information and successfully solve these tasks. To find a well-performing design, we present a computational design optimization algorithm and evaluate its effectiveness across different tasks and domains, showing promising results. Finally, we perform a human survey to evaluate the effectiveness of intuitive designs devised manually by humans, showing that the computationally found design is among the best designs in most cases.