VIANA: Visual Interactive Annotation of Argumentation

TL;DR

VIANA is a visual analytics system that enhances argumentation annotation by providing automatic suggestions, learning from user interactions, and integrating multiple views to improve efficiency and accuracy in argument mining tasks.

Contribution

The paper introduces a novel visual interactive system that combines automatic suggestion, semantic learning, and seamless visualization to improve argumentation annotation processes.

Findings

Experts preferred VIANA over existing solutions.

System provided significant speedup in annotation tasks.

Automatic suggestions improved over time with user interaction.

Abstract

Argumentation Mining addresses the challenging tasks of identifying boundaries of argumentative text fragments and extracting their relationships. Fully automated solutions do not reach satisfactory accuracy due to their insufficient incorporation of semantics and domain knowledge. Therefore, experts currently rely on time-consuming manual annotations. In this paper, we present a visual analytics system that augments the manual annotation process by automatically suggesting which text fragments to annotate next. The accuracy of those suggestions is improved over time by incorporating linguistic knowledge and language modeling to learn a measure of argument similarity from user interactions. Based on a long-term collaboration with domain experts, we identify and model five high-level analysis tasks. We enable close reading and note-taking, annotation of arguments, argument reconstruction, extraction of argument relations, and exploration of argument graphs. To avoid context switches, we transition between all views through seamless morphing, visually anchoring all text- and graph-based layers. We evaluate our system with a two-stage expert user study based on a corpus of presidential debates. The results show that experts prefer our system over existing solutions due to the speedup provided by the automatic suggestions and the tight integration between text and graph views.