# Human–AI collaboration: trade-offs between performance and preferences

## TL;DR

This study explores how humans prefer AI collaborators that consider their actions, leading to better teamwork and satisfaction.

## Contribution

The paper introduces an empirical framework to evaluate how algorithmic changes in AI agents affect human preferences and team performance.

## Key findings

- Agents that consider human actions are preferred over performance-maximizing agents.
- Human-centric AI design can improve likability without reducing performance.
- Inequality-aversion effects drive human preferences in AI collaboration.

## Abstract

Despite the growing interest in collaborative AI, designing systems that seamlessly integrate human input remains a major challenge. In this study, we developed a task to systematically examine human preferences for collaborative agents. We created and evaluated five collaborative AI agents with strategies that differ in the manner and degree they adapt to human actions. Participants interacted with a subset of these agents, evaluated their perceived traits, and selected their preferred agent. We used a Bayesian model to understand how agents’ strategies influence the human–AI team performance, AI’s perceived traits, and the factors shaping human preferences in pairwise agent comparisons. Our results show that agents who are more considerate of human actions are preferred over purely performance-maximizing agents. Moreover, we show that such human-centric design can improve the likability of AI collaborators without reducing performance. We find evidence for inequality-aversion effects being a driver of human choices, suggesting that people prefer collaborative agents which allow them to meaningfully contribute to the team. Taken together, these findings demonstrate how collaboration with AI can benefit from development efforts, which include both subjective and objective metrics.

Human-AI collaboration is expected to grow in the coming years. Particular attention is being paid to agentic cooperative AI that is capable of autonomously performing helpful tasks without repeated human instruction due to its potential to significantly improve the performance of human-AI teams. However, the use of cooperative AI agents poses two key challenges: (1) the development of such agents in modern multiagent reinforcement learning paradigms often excludes human collaborators, and (2) the process of integrating human preferences into the algorithms underlying AI agents remains poorly understood. Our study addresses these shortcomings by establishing an empirical framework to evaluate how algorithmic changes can be mapped to human preferences. Our study reveals key dynamics, such as algorithm changes that increase human liking of the AI agent without harming the performance of the human-AI team, and a pronounced human preference for inequity-aversion. These findings inform human-AI development by demonstrating how collaborative AI can be both effective and enjoyable. Our approach adjusts agent behavior by modifying algorithmic inputs and outputs, making it broadly applicable to new and existing agentic systems