SoDaCam: Software-defined Cameras via Single-Photon Imaging

TL;DR

SoDaCam introduces a flexible, software-defined camera system using single-photon imaging that can emulate various camera functionalities through photon-cube transformations, enabling advanced imaging capabilities and compression.

Contribution

It presents a novel approach to reinterpretable cameras at the photon level, leveraging photon-cube transformations for diverse imaging functionalities and efficient data compression.

Findings

Photon-cube transformations emulate multiple camera types.

The system achieves high-speed photon detection at 100 kHz.

Camera-dependent compression reduces data size effectively.

Abstract

Reinterpretable cameras are defined by their post-processing capabilities that exceed traditional imaging. We present "SoDaCam" that provides reinterpretable cameras at the granularity of photons, from photon-cubes acquired by single-photon devices. Photon-cubes represent the spatio-temporal detections of photons as a sequence of binary frames, at frame-rates as high as 100 kHz. We show that simple transformations of the photon-cube, or photon-cube projections, provide the functionality of numerous imaging systems including: exposure bracketing, flutter shutter cameras, video compressive systems, event cameras, and even cameras that move during exposure. Our photon-cube projections offer the flexibility of being software-defined constructs that are only limited by what is computable, and shot-noise. We exploit this flexibility to provide new capabilities for the emulated cameras. As an added benefit, our projections provide camera-dependent compression of photon-cubes, which we demonstrate using an implementation of our projections on a novel compute architecture that is designed for single-photon imaging.