Is It Possible to Make Chatbots Virtuous? Investigating a Virtue-Based Design Methodology Applied to LLMs

TL;DR

This paper explores a virtue-based design methodology for LLMs, cataloging ethical patterns and evaluating their impact on safety, robustness, and societal concerns through expert interviews.

Contribution

It applies a virtue-guided design approach to LLMs, creating new ethical patterns and assessing their practical implications and challenges.

Findings

Participants valued improved safety and robustness.

Concerns about jailbreaking and generalization.

Positive overall reaction with acknowledgment of tradeoffs.

Abstract

With the rapid growth of Large Language Models (LLMs), criticism of their societal impact has also grown. Work in Responsible AI (RAI) has focused on the development of AI systems aimed at reducing harm. Responding to RAI's criticisms and the need to bring the wisdom traditions into HCI, we apply Conwill et al.'s Virtue-Guided Technology Design method to LLMs. We cataloged new ethical design patterns for LLMs and evaluated them through interviews with technologists. Participants valued that the patterns provided more accuracy and robustness, better safety, new research opportunities, increased access and control, and reduced waste. Their concerns were that the patterns could be vulnerable to jailbreaking, were generalizing models too widely, and had potential implementation issues. Overall, participants reacted positively while also acknowledging the tradeoffs involved in ethical LLM design.