An approach to Physics Based Single Photon Vision System Design

TL;DR

This paper emphasizes the importance of incorporating photon noise models in the design of single-photon vision systems, demonstrating that optimal hardware depends on the specific vision task and noise characteristics.

Contribution

It introduces an end-to-end model for single-pixel cameras that highlights how non-linear photon noise influences optimal system design for various vision tasks.

Findings

Photon noise significantly impacts camera design choices.

Optimal hardware varies with the specific vision task.

Non-linear noise models improve system performance.

Abstract

The design of the camera and optical measurement is a crucial part of optimizing machine vision systems. However, camera designs are usually optimized to produce human-interpretable images. Moreover, camera optimization typically makes the assumption of additive noise, while modern optical imaging systems are mainly affected by photon noise which is not additive. Previous studies have highlighted the fundamental effect of the noise model on the outcome of the design process in the context of coded single-pixel and compressed sensing cameras. In addition to the exact noise model, the nature of the data encountered and the vision task fundamentally affect the optimal hardware design. This means that a camera or compressed sensing code that is optimal for capturing images is almost certainly not optimal for any downstream vision task. In this work, we used a simple end-to-end model on a single-pixel camera to study the effects of the noise model for different vision tasks. We demonstrate the importance of incorporating non-linear noise features in optimizing even the simplest vision systems.