SpaceEditing: Integrating Human Knowledge into Deep Neural Networks via Interactive Latent Space Editing

TL;DR

SpaceEditing is an interactive system that allows humans to modify the latent space of deep neural networks through visualized data manipulation, improving classification accuracy on ambiguous data.

Contribution

The paper introduces a novel interactive editing method and a new loss function to incorporate human knowledge into DNN training via latent space modification.

Findings

Effective in improving classification accuracy on ambiguous data

User-guided latent space editing enhances model interpretability

Demonstrated success through three case studies

Abstract

We propose an interactive editing method that allows humans to help deep neural networks (DNNs) learn a latent space more consistent with human knowledge, thereby improving classification accuracy on indistinguishable ambiguous data. Firstly, we visualize high-dimensional data features through dimensionality reduction methods and design an interactive system \textit{SpaceEditing} to display the visualized data. \textit{SpaceEditing} provides a 2D workspace based on the idea of spatial layout. In this workspace, the user can move the projection data in it according to the system guidance. Then, \textit{SpaceEditing} will find the corresponding high-dimensional features according to the projection data moved by the user, and feed the high-dimensional features back to the network for retraining, therefore achieving the purpose of interactively modifying the high-dimensional latent space for the user. Secondly, to more rationally incorporate human knowledge into the training process of neural networks, we design a new loss function that enables the network to learn user-modified information. Finally, We demonstrate how \textit{SpaceEditing} meets user needs through three case studies while evaluating our proposed new method, and the results confirm the effectiveness of our method.