Ternary-Input Binary-Weight CNN Accelerator Design for Miniature Object Classification System with Query-Driven Spatial DVS

TL;DR

This paper introduces a specialized CNN accelerator for miniature object classification systems that leverages ternary DVS data and binary weights to significantly reduce power, computation, and memory requirements, enabling efficient real-time processing.

Contribution

It presents a novel reconfigurable sensor and a ternary-input, binary-weight neural network hardware accelerator optimized for miniature imaging systems.

Findings

Reduces data size by 81%

Cuts MAC operations by 27%

Achieves 440 ms inference at 1.6 mW

Abstract

Miniature imaging systems are essential for space-constrained applications but are limited by memory and power constraints. While machine learning can reduce data size by extracting key features, its high energy demands often exceed the capacity of small batteries. This paper presents a CNN hardware accelerator optimized for object classification in miniature imaging systems. It processes data from a spatial Dynamic Vision Sensor (DVS), reconfigurable to a temporal DVS via pixel sharing, minimizing sensor area. By using ternary DVS outputs and a ternary-input, binary-weight neural network, the design reduces computation and memory needs. Fabricated in 28 nm CMOS, the accelerator cuts data size by 81% and MAC operations by 27%. It achieves 440 ms inference time at just 1.6 mW power consumption, improving the Figure-of-Merit (FoM) by 7.3x over prior CNN accelerators for miniature systems.