A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture

TL;DR

This paper introduces a reconfigurable convolution-in-pixel CMOS image sensor architecture that enhances processing speed and power efficiency for AI applications at the sensor level, addressing fill-factor issues of traditional PIP schemes.

Contribution

It proposes a novel PIP CMOS sensor architecture enabling convolution before readout, significantly improving speed, efficiency, and fill-factor over conventional designs.

Findings

Supports up to 11.65 TOPS/W efficiency at 8-bit weights

Uses only 2.5 transistors per pixel, boosting fill-factor

Achieves three times higher efficiency than conventional schemes

Abstract

The separation of the data capture and analysis in modern vision systems has led to a massive amount of data transfer between the end devices and cloud computers, resulting in long latency, slow response, and high power consumption. Efficient hardware architectures are under focused development to enable Artificial Intelligence (AI) at the resource-limited end sensing devices. One of the most promising solutions is to enable Processing-in-Pixel (PIP) scheme. However, the conventional schemes suffer from the low fill-factor issue. This paper proposes a PIP based CMOS sensor architecture, which allows convolution operation before the column readout circuit to significantly improve the image reading speed with much lower power consumption. The simulation results show that the proposed architecture could support the computing efficiency up to 11.65 TOPS/W at the 8-bit weight configuration, which is three times as high as the conventional schemes. The transistors required for each pixel are only 2.5T, significantly improving the fill-factor.