All-Optical Segmentation via Diffractive Neural Networks for Autonomous Driving

TL;DR

This paper introduces an all-optical neural network system that performs semantic segmentation and lane detection for autonomous driving, offering energy-efficient and high-speed processing by leveraging light diffraction.

Contribution

The work presents a novel all-optical computing framework for image segmentation and lane detection, demonstrating its effectiveness and generalizability in autonomous driving scenarios.

Findings

Effective segmentation on CityScapes dataset

Successful lane detection in simulated driving scenarios

Reduced energy consumption compared to traditional DNNs

Abstract

Semantic segmentation and lane detection are crucial tasks in autonomous driving systems. Conventional approaches predominantly rely on deep neural networks (DNNs), which incur high energy costs due to extensive analog-to-digital conversions and large-scale image computations required for low-latency, real-time responses. Diffractive optical neural networks (DONNs) have shown promising advantages over conventional DNNs on digital or optoelectronic computing platforms in energy efficiency. By performing all-optical image processing via light diffraction at the speed of light, DONNs save computation energy costs while reducing the overhead associated with analog-to-digital conversions by all-optical encoding and computing. In this work, we propose a novel all-optical computing framework for RGB image segmentation and lane detection in autonomous driving applications. Our experimental results demonstrate the effectiveness of the DONN system for image segmentation on the CityScapes dataset. Additionally, we conduct case studies on lane detection using a customized indoor track dataset and simulated driving scenarios in CARLA, where we further evaluate the model's generalizability under diverse environmental conditions.