A Deep Generative Model for Interactive Data Annotation through Direct Manipulation in Latent Space

TL;DR

This paper introduces a deep generative model that enables interactive data annotation by directly manipulating latent space, incorporating new analogies like time and force, and analyzing hyperparameter effects on data representation.

Contribution

It proposes novel analogies for interaction, a network model for compact data representation considering structure and annotations, and studies hyperparameter impacts for improved annotation tools.

Findings

New analogies for interaction: time and force.

A model for learning data representations with annotations.

Hyperparameter effects on data representation identified.

Abstract

The impact of machine learning (ML) in many fields of application is constrained by lack of annotated data. Among existing tools for ML-assisted data annotation, one little explored tool type relies on an analogy between the coordinates of a graphical user interface and the latent space of a neural network for interaction through direct manipulation. In the present work, we 1) expand the paradigm by proposing two new analogies: time and force as reflecting iterations and gradients of network training; 2) propose a network model for learning a compact graphical representation of the data that takes into account both its internal structure and user provided annotations; and 3) investigate the impact of model hyperparameters on the learned graphical representations of the data, identifying candidate model variants for a future user study.