PersonaBOT: Bringing Customer Personas to Life with LLMs and RAG

TL;DR

This paper presents a method to generate synthetic customer personas using LLMs and integrate them into a RAG chatbot to enhance decision-making support, demonstrating improved accuracy and utility in business scenarios.

Contribution

It introduces a novel approach combining Few-Shot and Chain-of-Thought prompting to generate synthetic personas for RAG chatbots, improving scalability and response quality.

Findings

Few-Shot prompting yields more complete personas.

Chain-of-Thought is more efficient in response time.

Chatbot accuracy improved from 5.88 to 6.42 out of 10.

Abstract

The introduction of Large Language Models (LLMs) has significantly transformed Natural Language Processing (NLP) applications by enabling more advanced analysis of customer personas. At Volvo Construction Equipment (VCE), customer personas have traditionally been developed through qualitative methods, which are time-consuming and lack scalability. The main objective of this paper is to generate synthetic customer personas and integrate them into a Retrieval-Augmented Generation (RAG) chatbot to support decision-making in business processes. To this end, we first focus on developing a persona-based RAG chatbot integrated with verified personas. Next, synthetic personas are generated using Few-Shot and Chain-of-Thought (CoT) prompting techniques and evaluated based on completeness, relevance, and consistency using McNemar's test. In the final step, the chatbot's knowledge base is augmented with synthetic personas and additional segment information to assess improvements in response accuracy and practical utility. Key findings indicate that Few-Shot prompting outperformed CoT in generating more complete personas, while CoT demonstrated greater efficiency in terms of response time and token usage. After augmenting the knowledge base, the average accuracy rating of the chatbot increased from 5.88 to 6.42 on a 10-point scale, and 81.82% of participants found the updated system useful in business contexts.