DAD vision: opto-electronic co-designed computer vision with division adjoint method

TL;DR

This paper introduces an opto-electronic co-design approach using ultra-thin diffractive optical elements to perform passive optical convolution, significantly reducing computational load in computer vision systems.

Contribution

It presents a novel division adjoint co-design method and demonstrates replacing initial neural network layers with optical convolution without power consumption.

Findings

Optical convolution achieves similar accuracy to digital in CIFAR-10

Passive optical components reduce power consumption in vision systems

First layers of neural networks can be replaced by optical elements

Abstract

The miniaturization and mobility of computer vision systems are limited by the heavy computational burden and the size of optical lenses. Here, we propose to use a ultra-thin diffractive optical element to implement passive optical convolution. A division adjoint opto-electronic co-design method is also proposed. In our simulation experiments, the first few convolutional layers of the neural network can be replaced by optical convolution in a classification task on the CIFAR-10 dataset with no power consumption, while similar performance can be obtained.