Scalable Coherent Optical Crossbar Architecture using PCM for AI Acceleration

TL;DR

This paper proposes a scalable optical AI accelerator architecture using phase change material for weight storage, modeled with silicon photonics, achieving high inference speeds with significantly lower power and area compared to GPUs.

Contribution

It introduces a novel scalable crossbar optical architecture with PCM-based weights, system-level modeling, and analysis for AI acceleration in datacenters.

Findings

Achieves inference speeds comparable to Nvidia A100 GPU.

Reduces power consumption by 15.4 times.

Reduces area by 7.24 times.

Abstract

Optical computing has been recently proposed as a new compute paradigm to meet the demands of future AI/ML workloads in datacenters and supercomputers. However, proposed implementations so far suffer from lack of scalability, large footprints and high power consumption, and incomplete system-level architectures to become integrated within existing datacenter architecture for real-world applications. In this work, we present a truly scalable optical AI accelerator based on a crossbar architecture. We have considered all major roadblocks and address them in this design. Weights will be stored on chip using phase change material (PCM) that can be monolithically integrated in silicon photonic processes. All electro-optical components and circuit blocks are modeled based on measured performance metrics in a 45nm monolithic silicon photonic process, which can be co-packaged with advanced CPU/GPUs and HBM memories. We also present a system-level modeling and analysis of our chip's performance for the Resnet-50V1.5, considering all critical parameters, including memory size, array size, photonic losses, and energy consumption of peripheral electronics. Both on-chip SRAM and off-chip DRAM energy overheads have been considered in this modeling. We additionally address how using a dual-core crossbar design can eliminate programming time overhead at practical SRAM block sizes and batch sizes. Our results show that a 128 x 128 proposed architecture can achieve inference per second (IPS) similar to Nvidia A100 GPU at 15.4 times lower power and 7.24 times lower area.