End-to-end image compression and reconstruction with ultrahigh speed and ultralow energy enabled by opto-electronic computing processor

TL;DR

This paper introduces a novel optoelectronic computing processor that achieves ultra-fast, low-energy image compression and reconstruction, surpassing electronic processors in speed and efficiency, suitable for real-time applications like aerial imagery processing.

Contribution

The work presents the first end-to-end image compression and reconstruction system using a silicon photonic chip with programmable matrices, enabling adjustable compression ratios and real-time processing.

Findings

Achieves 49.5 ps/pixel latency and less than 10.6 nJ/pixel energy consumption.

Demonstrates real-time compression of 130 million-pixel aerial images.

Outperforms GPU-based models by 2-3 orders of magnitude in speed and energy efficiency.

Abstract

The rapid development of AR/VR, remote sensing, satellite radar, and medical equipment has created an imperative demand for ultra efficient image compression and reconstruction that exceed the capabilities of electronic processors. For the first time, we demonstrate an end to end image compression and reconstruction approach using an optoelectronic computing processor,achieving orders of magnitude higher speed and lower energy consumption than electronic counterparts. At its core is a 32X32 silicon photonic computing chip, which monolithically integrates 32 high speed modulators, 32 detectors, and a programmable photonic matrix core, copackaged with all necessary control electronics (TIA, ADC, DAC, FPGA etc.). Leveraging the photonic matrix core programmability, the processor generates trainable compressive matrices, enabling adjustable image compression ratios (from 2X to 256X) to meet diverse application needs. Deploying a custom lightweight photonic integrated circuit oriented network (LiPICO-Net) enables high quality reconstruction of compressed images. Our approach delivers an end to end latency of only 49.5ps/pixel while consuming only less than 10.6nJ/pixel-both metrics representing 2-3 orders of magnitude improvement compared with classical models running on state-of-the-art GPUs. We validate the system on a 130 million-pixel aerial imagery, enabling real time compression where electronic systems falter due to power and latency constraints. This work not only provides a transformative solution for massive image processing but also opens new avenues for photonic computing applications.