Forecasting Live Chat Intent from Browsing History

TL;DR

This paper introduces a two-stage method to predict detailed user intents in live chat scenarios from browsing history, combining intent classification with language models for fine-grained intent generation, improving accuracy over direct methods.

Contribution

The paper presents a novel two-stage approach that first classifies browsing history into high-level intents and then uses LLMs to generate detailed intents, enhancing prediction performance.

Findings

Significant performance improvements over direct intent generation methods.

Effective use of pretrained Transformers for intent classification.

Automatic evaluation aligns well with human judgment.

Abstract

Customers reach out to online live chat agents with various intents, such as asking about product details or requesting a return. In this paper, we propose the problem of predicting user intent from browsing history and address it through a two-stage approach. The first stage classifies a user's browsing history into high-level intent categories. Here, we represent each browsing history as a text sequence of page attributes and use the ground-truth class labels to fine-tune pretrained Transformers. The second stage provides a large language model (LLM) with the browsing history and predicted intent class to generate fine-grained intents. For automatic evaluation, we use a separate LLM to judge the similarity between generated and ground-truth intents, which closely aligns with human judgments. Our two-stage approach yields significant performance gains compared to generating intents without the classification stage.