HEDS 3.0: The Human Evaluation Data Sheet Version 3.0

TL;DR

HEDS 3.0 introduces an improved, user-friendly version of the human evaluation datasheet, facilitating more consistent and comparable data collection in human evaluation experiments.

Contribution

It provides an updated, clearer datasheet version with accompanying documentation and code, based on extensive user feedback and practical experience.

Findings

Enhanced clarity and usability of the datasheet

Improved consistency in human evaluation data collection

Accessible tools for exporting data as LaTeX files

Abstract

This paper presents version 3.0 of the Human Evaluation Datasheet (HEDS). This update is the result of our experience using HEDS in the context of numerous recent human evaluation experiments, including reproduction studies, and of feedback received. Our main overall goal was to improve clarity, and to enable users to complete the datasheet more consistently and comparably. The HEDS 3.0 package consists of the digital data sheet, documentation, and code for exporting completed data sheets as latex files, all available from the HEDS GitHub.