On-Device Super Resolution Imaging Using Low-Cost SPAD Array and Embedded Lightweight Deep Learning

TL;DR

This paper introduces LiteSR, a lightweight neural network that enhances low-resolution SPAD sensor images to high-resolution in real-time, using embedded deep learning on a microcontroller for cost-effective, scalable super-resolution imaging.

Contribution

The work presents a novel embedded deep learning framework for real-time super-resolution of SPAD sensor images, demonstrating high fidelity and robustness on synthetic and real datasets.

Findings

Achieves high-quality 256x256 super-resolution from low-res SPAD images.

Demonstrates real-time SR video streaming with embedded microcontroller.

Extends up to 512x512 resolution, even with noisy inputs.

Abstract

This work presents a lightweight super-resolution (LiteSR) neural network for depth and intensity images acquired from a consumer-grade single-photon avalanche diode (SPAD) array with a 48x32 spatial resolution. The proposed framework reconstructs high-resolution (HR) images of size 256x256. Both synthetic and real datasets are used for performance evaluation. Extensive quantitative metrics demonstrate high reconstruction fidelity on synthetic datasets, while experiments on real indoor and outdoor measurements further confirm the robustness of the proposed approach. Moreover, the SPAD sensor is interfaced with an Arduino UNO Q microcontroller, which receives low-resolution (LR) depth and intensity images and feeds them into a compressed, pre-trained deep learning (DL) model, enabling real-time SR video streaming. In addition to the 256x256 setting, a range of target HR resolutions is evaluated to determine the maximum achievable upscaling resolution (512x512) with LiteSR, including scenarios with noise-corrupted LR inputs. The proposed LiteSR-embedded system co-design provides a scalable, cost-effective solution to enhance the spatial resolution of current consumer-grade SPAD arrays to meet HR imaging requirements.