# Integrated photonic 3D tensor processing engine

## TL;DR

This paper introduces a photonic processor that efficiently handles 3D tensor operations for deep learning, achieving high accuracy in LiDAR image recognition.

## Contribution

The novel 3D tensor processing engine integrates optical caching, synchronization, and computation for high-order tensor convolutions.

## Key findings

- The 3D-TPE operates at clock frequencies from 10 GHz to 30 GHz.
- It achieves 97.06% accuracy in LiDAR 3D point cloud classification.
- Optical components reduce memory and time overheads compared to electrical reshaping.

## Abstract

Optical computing leverages high bandwidth, low latency, and power efficiency, which is considered as one of the most effective solutions for accelerating deep learning tasks. However, mainstream photonic hardware accelerators are primarily optimized for two-dimensional (2D) matrix-vector multiplications (MVMs). To implement three-dimensional (3D) convolutional neural networks (CNNs), high-order tensors must be reshaped in the electrical domain according to the size of the accelerators before computation, leading to extra memory usage and time overheads. Additionally, synchronization across multiple channels depends on external electronic clocks, which increases the complexity of the system. In this work, we propose an integrated photonic 3D tensor processing engine (3D-TPE) based on the interleaving modulation of time, wavelength, and space. Data caching, channel synchronization and computation are realized entirely within the optical domain, reducing memory and time usage, and simplifying the system. Optical caching and synchronization are achieved with an optical tunable delay line (OTDL) chip supporting versatile clock frequencies up to 200 GHz, and optical computing is accomplished with a dual-coupled micro-ring resonators (MRRs) based crossbar chip with a 3-dB passband width of 50 GHz. We verify the processing capabilities of the 3D-TPE at clock frequencies ranging from 10 GHz to 30 GHz and perform a proof-of-concept experiment for a LiDAR 3D point cloud image recognition task operating at 20 GHz, achieving a recognition accuracy of 97.06%. The proposed 3D-TPE is anticipated to facilitate high-order tensor convolutions, playing an important role in autonomous driving, healthcare, video analytics, virtual reality, etc.

An integrated photonic 3D tensor processing engine enables 3D tensor computation, caching, and synchronization with tunable clock frequencies, experimentally achieving operation from 10 GHz to 30 GHz and 97.06% LiDAR point-cloud classification accuracy.