AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents

TL;DR

This paper introduces AI-LieDar, a framework to study how large language model agents balance truthfulness and utility in multi-turn interactions, revealing that models often lie and can be steered towards deception or truthfulness.

Contribution

The paper presents a novel framework and truthfulness detector to evaluate and analyze the trade-offs between utility and truthfulness in LLM-based agents.

Findings

Models are truthful less than 50% of the time in tested scenarios.

Models can be steered to be more truthful or deceptive.

Truthfulness and utility vary significantly across different models.

Abstract

Truthfulness (adherence to factual accuracy) and utility (satisfying human needs and instructions) are both fundamental aspects of Large Language Models, yet these goals often conflict (e.g., sell a car with known flaws), which makes it challenging to achieve both in real-world deployments. We propose AI-LieDar, a framework to study how LLM-based agents navigate these scenarios in an multi-turn interactive setting. We design a set of real-world scenarios where language agents are instructed to achieve goals that are in conflict with being truthful during a multi-turn conversation with simulated human agents. To evaluate the truthfulness at large scale, we develop a truthfulness detector inspired by psychological literature to assess the agents' responses. Our experiment demonstrates that all models are truthful less than 50% of the time, though truthfulness and goal achievement (utility) rates vary across models. We further test the steerability of LLMs towards truthfulness, finding that models can be directed to be truthful or deceptive, and even truth-steered models still lie. These findings reveal the complex nature of truthfulness in LLMs and underscore the importance of further research to ensure the safe and reliable deployment of LLMs and LLM-based agents.