Debt Collection Negotiations with Large Language Models: An Evaluation System and Optimizing Decision Making with Multi-Agent

TL;DR

This paper evaluates the use of large language models in automating debt collection negotiations, introduces a comprehensive evaluation framework, and proposes a multi-agent system to improve decision-making and negotiation outcomes.

Contribution

It presents a novel multi-agent framework with planning and judging modules for dynamic debt negotiations and introduces a new evaluation framework with 13 metrics.

Findings

LLMs tend to over-concede compared to humans

The MADeN framework improves decision rationality

Post-training techniques enhance LLM negotiation performance

Abstract

Debt collection negotiations (DCN) are vital for managing non-performing loans (NPLs) and reducing creditor losses. Traditional methods are labor-intensive, while large language models (LLMs) offer promising automation potential. However, prior systems lacked dynamic negotiation and real-time decision-making capabilities. This paper explores LLMs in automating DCN and proposes a novel evaluation framework with 13 metrics across 4 aspects. Our experiments reveal that LLMs tend to over-concede compared to human negotiators. To address this, we propose the Multi-Agent Debt Negotiation (MADeN) framework, incorporating planning and judging modules to improve decision rationality. We also apply post-training techniques, including DPO with rejection sampling, to optimize performance. Our studies provide valuable insights for practitioners and researchers seeking to enhance efficiency and outcomes in this domain.