Methodology to Deploy CNN-Based Computer Vision Models on Immersive Wearable Devices

TL;DR

This paper introduces a method for deploying CNN models on AR headsets by training on computers and transferring optimized weights, enabling real-time image recognition with maintained accuracy.

Contribution

It presents a novel transfer approach that allows CNN models trained on traditional hardware to run effectively on resource-constrained AR headsets.

Findings

CNN models can be transferred to AR headsets with minimal accuracy loss

Real-time image recognition is feasible on AR devices using this method

The approach maintains about 98% accuracy on the MNIST dataset

Abstract

Convolutional Neural Network (CNN) models often lack the ability to incorporate human input, which can be addressed by Augmented Reality (AR) headsets. However, current AR headsets face limitations in processing power, which has prevented researchers from performing real-time, complex image recognition tasks using CNNs in AR headsets. This paper presents a method to deploy CNN models on AR headsets by training them on computers and transferring the optimized weight matrices to the headset. The approach transforms the image data and CNN layers into a one-dimensional format suitable for the AR platform. We demonstrate this method by training the LeNet-5 CNN model on the MNIST dataset using PyTorch and deploying it on a HoloLens AR headset. The results show that the model maintains an accuracy of approximately 98%, similar to its performance on a computer. This integration of CNN and AR enables real-time image processing on AR headsets, allowing for the incorporation of human input into AI models.