Machine-learning enables Image Reconstruction and Classification in a "see-through" camera

TL;DR

This paper shows that convolutional neural networks can reconstruct images and classify data directly from raw sensor inputs in a transparent camera setup, achieving comparable accuracy to traditional image-based methods.

Contribution

It introduces a neural network approach for image reconstruction and classification directly from raw sensor data in a see-through camera system, demonstrating potential for real-time applications.

Findings

Reconstructed images enable effective classification with neural networks.

Classification accuracy from raw sensor data is comparable to that from reconstructed images.

The method works across multiple datasets including MNIST, EMNIST, and Kanji49.

Abstract

We demonstrate that image reconstruction can be achieved via a convolutional neural network for a "see-through" computational camera comprised of a transparent window and a CMOS image sensor. Furthermore, we compared classification results using a classifier network for the raw sensor data vs the reconstructed images. The results suggest that similar classification accuracy is likely possible in both cases with appropriate network optimizations. All networks were trained and tested for the MNIST (6 classes), EMNIST and the Kanji49 datasets.