Hierarchical Tree Search-based User Lifelong Behavior Modeling on Large Language Model

TL;DR

This paper introduces HiT-LBM, a hierarchical framework that improves user lifelong behavior modeling for recommendation systems by effectively capturing interest evolution from large-scale sequential user data using tree search and interest fusion.

Contribution

The paper proposes a novel hierarchical tree search framework with chunked behavior extraction and interest fusion to better model user interests from extensive sequential data.

Findings

Outperforms state-of-the-art methods in experiments

Effectively captures interest evolution over time

Enhances recommendation performance with comprehensive user interest representation

Abstract

Large Language Models (LLMs) have garnered significant attention in Recommendation Systems (RS) due to their extensive world knowledge and robust reasoning capabilities. However, a critical challenge lies in enabling LLMs to effectively comprehend and extract insights from massive user behaviors. Current approaches that directly leverage LLMs for user interest learning face limitations in handling long sequential behaviors, effectively extracting interest, and applying interest in practical scenarios. To address these issues, we propose a Hierarchical Tree Search-based User Lifelong Behavior Modeling framework (HiT-LBM). HiT-LBM integrates Chunked User Behavior Extraction (CUBE) and Hierarchical Tree Search for Interest (HTS) to capture diverse interests and interest evolution of user. CUBE divides user lifelong behaviors into multiple chunks and learns the interest and interest evolution within each chunk in a cascading manner. HTS generates candidate interests through hierarchical expansion and searches for the optimal interest with process rating model to ensure information gain for each behavior chunk. Additionally, we design Temporal-Ware Interest Fusion (TIF) to integrate interests from multiple behavior chunks, constructing a comprehensive representation of user lifelong interests. The representation can be embedded into any recommendation model to enhance performance. Extensive experiments demonstrate the effectiveness of our approach, showing that it surpasses state-of-the-art methods.