A Virtual Reality Tool for Representing, Visualizing and Updating Deep Learning Models

TL;DR

This paper presents a virtual reality tool that visualizes and interacts with deep learning models, making their inner workings tangible and transparent to users, thereby enhancing understanding and development.

Contribution

The paper introduces a VR-based visualization method that maps neural network concepts to virtual space, enabling intuitive exploration and understanding of deep learning models.

Findings

The VR tool effectively visualizes data clusters and model training dynamics.

Users can interactively categorize data, observing real-time model updates.

The approach enhances transparency and interpretability of deep neural networks.

Abstract

Deep learning is ubiquitous, but its lack of transparency limits its impact on several potential application areas. We demonstrate a virtual reality tool for automating the process of assigning data inputs to different categories. A dataset is represented as a cloud of points in virtual space. The user explores the cloud through movement and uses hand gestures to categorise portions of the cloud. This triggers gradual movements in the cloud: points of the same category are attracted to each other, different groups are pushed apart, while points are globally distributed in a way that utilises the entire space. The space, time, and forces observed in virtual reality can be mapped to well-defined machine learning concepts, namely the latent space, the training epochs and the backpropagation. Our tool illustrates how the inner workings of deep neural networks can be made tangible and transparent. We expect this approach to accelerate the autonomous development of deep learning applications by end users in novel areas.