Beyond Accuracy: Assessing Software Documentation Quality

TL;DR

This paper proposes a comprehensive framework to evaluate software documentation quality across ten dimensions, aiming to clarify what constitutes 'good' documentation and improve assessment methods.

Contribution

It introduces a novel, structured framework for assessing software documentation quality based on dimensions of structure, content, and style, validated through expert evaluations.

Findings

Reference documentation and README files generally have higher quality.

Blog articles tend to have more issues in documentation quality.

The framework enables more systematic and nuanced quality assessments.

Abstract

Good software documentation encourages good software engineering, but the meaning of "good" documentation is vaguely defined in the software engineering literature. To clarify this ambiguity, we draw on work from the data and information quality community to propose a framework that decomposes documentation quality into ten dimensions of structure, content, and style. To demonstrate its application, we recruited technical editors to apply the framework when evaluating examples from several genres of software documentation. We summarise their assessments -- for example, reference documentation and README files excel in quality whereas blog articles have more problems -- and we describe our vision for reasoning about software documentation quality and for the expansion and potential of a unified quality framework.