AI-Salesman: Towards Reliable Large Language Model Driven Telemarketing

TL;DR

This paper introduces AI-Salesman, a framework for reliable, goal-driven telemarketing dialogue using large language models, featuring a new dataset, a dual-stage architecture, and a comprehensive evaluation method.

Contribution

It presents a novel dual-stage framework with Bayesian reinforcement learning and dynamic strategic guidance, along with the first real-world grounded telemarketing dialogue dataset.

Findings

AI-Salesman outperforms baselines in automatic metrics.

Demonstrates effectiveness in complex persuasive scenarios.

Introduces a new dataset for telemarketing dialogues.

Abstract

Goal-driven persuasive dialogue, exemplified by applications like telemarketing, requires sophisticated multi-turn planning and strict factual faithfulness, which remains a significant challenge for even state-of-the-art Large Language Models (LLMs). A lack of task-specific data often limits previous works, and direct LLM application suffers from strategic brittleness and factual hallucination. In this paper, we first construct and release TeleSalesCorpus, the first real-world-grounded dialogue dataset for this domain. We then propose AI-Salesman, a novel framework featuring a dual-stage architecture. For the training stage, we design a Bayesian-supervised reinforcement learning algorithm that learns robust sales strategies from noisy dialogues. For the inference stage, we introduce the Dynamic Outline-Guided Agent (DOGA), which leverages a pre-built script library to provide dynamic, turn-by-turn strategic guidance. Moreover, we design a comprehensive evaluation framework that combines fine-grained metrics for key sales skills with the LLM-as-a-Judge paradigm. Experimental results demonstrate that our proposed AI-Salesman significantly outperforms baseline models in both automatic metrics and comprehensive human evaluations, showcasing its effectiveness in complex persuasive scenarios.