Use of Formal Ethical Reviews in NLP Literature: Historical Trends and Current Practices

TL;DR

This paper analyzes the historical and current trends of formal ethical reviews in NLP research, comparing it with related fields to understand the evolution of ethical awareness.

Contribution

It provides a comprehensive quantitative and qualitative analysis of ethical review practices in NLP literature and compares these trends across related disciplines.

Findings

Increase in mention of ethical approvals over time

NLP shows similar ethical review trends as related fields

Greater awareness of ethical issues in recent NLP research

Abstract

Ethical aspects of research in language technologies have received much attention recently. It is a standard practice to get a study involving human subjects reviewed and approved by a professional ethics committee/board of the institution. How commonly do we see mention of ethical approvals in NLP research? What types of research or aspects of studies are usually subject to such reviews? With the rising concerns and discourse around the ethics of NLP, do we also observe a rise in formal ethical reviews of NLP studies? And, if so, would this imply that there is a heightened awareness of ethical issues that was previously lacking? We aim to address these questions by conducting a detailed quantitative and qualitative analysis of the ACL Anthology, as well as comparing the trends in our field to those of other related disciplines, such as cognitive science, machine learning, data mining, and systems.