Low-power In-pixel Computing with Current-modulated Switched Capacitors

TL;DR

This paper introduces a scalable in-pixel processing architecture using current-modulated switched capacitors and PWM, significantly reducing data throughput and power consumption while maintaining high accuracy for object classification.

Contribution

The proposed architecture employs a non-destructive, PWM-based switched capacitor approach that decouples sensor exposure from processing, enabling high data fidelity and parallelism in in-pixel computing.

Findings

Reduces data throughput by 10X

Consumes less than 30 mW per megapixel

Performs comparably to CNN-based methods

Abstract

We present a scalable in-pixel processing architecture that can reduce the data throughput by 10X and consume less than 30 mW per megapixel at the imager frontend. Unlike the state-of-the-art (SOA) analog process-in-pixel (PIP) that modulates the exposure time of photosensors when performing matrix-vector multiplications, we use switched capacitors and pulse width modulation (PWM). This non-destructive approach decouples the sensor exposure and computing, providing processing parallelism and high data fidelity. Our design minimizes the computational complexity and chip density by leveraging the patch-based feature extraction that can perform as well as the CNN. We further reduce data using partial observation of the attended objects, which performs closely to the full frame observations. We have been studying the reduction of output features as a function of accuracy, chip density and power consumption from a transformer-based backend model for object classification and detection.