A Computational Analysis of Vagueness in Revisions of Instructional Texts

TL;DR

This paper analyzes how vagueness in instructional texts on WikiHow is revised, and evaluates a neural model's ability to distinguish between different versions of instructions, highlighting improvements over previous methods.

Contribution

It introduces a method to extract and analyze revisions involving vagueness in instructional texts and demonstrates a neural model's effectiveness in distinguishing instruction versions.

Findings

Neural model outperforms baselines in pairwise ranking tasks.

Vagueness in instructions can be systematically identified and analyzed.

Revisions often reduce vagueness, improving clarity.

Abstract

WikiHow is an open-domain repository of instructional articles for a variety of tasks, which can be revised by users. In this paper, we extract pairwise versions of an instruction before and after a revision was made. Starting from a noisy dataset of revision histories, we specifically extract and analyze edits that involve cases of vagueness in instructions. We further investigate the ability of a neural model to distinguish between two versions of an instruction in our data by adopting a pairwise ranking task from previous work and showing improvements over existing baselines.