Democratizing Ethical Assessment of Natural Language Generation Models

TL;DR

This paper introduces TEAL, an open-source tool designed to make ethical assessment of natural language generation models more accessible and standardized for broader use.

Contribution

The paper presents TEAL, a novel tool that democratizes and standardizes ethical evaluation of language models, addressing high entry barriers in the field.

Findings

TEAL enables broader access to ethical assessments.

TEAL standardizes evaluation processes.

Improves transparency in language model ethics.

Abstract

Natural language generation models are computer systems that generate coherent language when prompted with a sequence of words as context. Despite their ubiquity and many beneficial applications, language generation models also have the potential to inflict social harms by generating discriminatory language, hateful speech, profane content, and other harmful material. Ethical assessment of these models is therefore critical. But it is also a challenging task, requiring an expertise in several specialized domains, such as computational linguistics and social justice. While significant strides have been made by the research community in this domain, accessibility of such ethical assessments to the wider population is limited due to the high entry barriers. This article introduces a new tool to democratize and standardize ethical assessment of natural language generation models: Tool for Ethical Assessment of Language generation models (TEAL), a component of Credo AI Lens, an open-source assessment framework.