Multi-Agent LLMs as Ethics Advocates for AI-Based Systems

TL;DR

This paper introduces a multi-agent LLM framework with an ethics advocate agent to assist in generating ethics requirements for AI systems, aiming to improve ethical considerations in requirements engineering.

Contribution

It presents a novel multi-agent LLM-based framework that critiques and generates ethics requirements, addressing challenges in stakeholder input and resource constraints.

Findings

Captures most ethics requirements from interviews within 30 minutes

Identifies additional relevant ethics requirements beyond interviews

Highlights reliability issues requiring human feedback

Abstract

Incorporating ethics into the requirement elicitation process is essential for creating ethically aligned systems. Although eliciting manual ethics requirements is effective, it requires diverse input from multiple stakeholders, which can be challenging due to time and resource constraints. Moreover, it is often given a low priority in the requirements elicitation process. This study proposes a framework for generating ethics requirements drafts by introducing an ethics advocate agent in a multi-agent LLM setting. This agent critiques and provides input on ethical issues based on the system description. The proposed framework is evaluated through two case studies from different contexts, demonstrating that it captures the majority of ethics requirements identified by researchers during 30-minute interviews and introduces several additional relevant requirements. However, it also highlights reliability issues in generating ethics requirements, emphasizing the need for human feedback in this sensitive domain. We believe this work can facilitate the broader adoption of ethics in the requirements engineering process, ultimately leading to more ethically aligned products.