End-to-End Framework for Efficient Deep Learning Using Metasurfaces Optics

TL;DR

This paper introduces a novel optical framework using metasurfaces to perform CNN computations directly in free-space, enabling efficient RGB data processing with significant energy savings and minimal accuracy loss.

Contribution

It presents the first general metasurface-based optical system capable of processing RGB images directly, advancing beyond single-channel limitations of previous approaches.

Findings

Achieves up to tenfold energy savings

Simplifies sensor design significantly

Maintains high network accuracy

Abstract

Deep learning using Convolutional Neural Networks (CNNs) has been shown to significantly out-performed many conventional vision algorithms. Despite efforts to increase the CNN efficiency both algorithmically and with specialized hardware, deep learning remains difficult to deploy in resource-constrained environments. In this paper, we propose an end-to-end framework to explore optically compute the CNNs in free-space, much like a computational camera. Compared to existing free-space optics-based approaches which are limited to processing single-channel (i.e., grayscale) inputs, we propose the first general approach, based on nanoscale meta-surface optics, that can process RGB data directly from the natural scenes. Our system achieves up to an order of magnitude energy saving, simplifies the sensor design, all the while sacrificing little network accuracy.