LumiCRS: Asymmetric Contrastive Prototype Learning for Long-Tail Conversational Recommender Systems

TL;DR

LumiCRS introduces an innovative framework that addresses long-tail distribution challenges in conversational recommender systems by combining adaptive loss, prototype learning, and dialogue augmentation, significantly improving diversity and fairness.

Contribution

The paper proposes LumiCRS, a novel end-to-end approach that mitigates long-tail issues in CRSs through adaptive loss, prototype-based clustering, and GPT-4o-driven dialogue augmentation, enhancing recommendation quality.

Findings

Boosts Recall@10 and Tail-Recall@10 by 7-15% on benchmarks.

Improves diversity, fairness, and long-tail relevance in recommendations.

Human evaluations show enhanced fluency and informativeness.

Abstract

Conversational recommender systems (CRSs) often suffer from an extreme long-tail distribution of dialogue data, causing a strong bias toward head-frequency blockbusters that sacrifices diversity and exacerbates the cold-start problem. An empirical analysis of DCRS and statistics on the REDIAL corpus show that only 10% of head movies account for nearly half of all mentions, whereas about 70% of tail movies receive merely 26% of the attention. This imbalance gives rise to three critical challenges: head over-fitting, body representation drift, and tail sparsity. To address these issues, we propose LumiCRS, an end-to-end framework that mitigates long-tail imbalance through three mutually reinforcing layers: (i) an Adaptive Comprehensive Focal Loss (ACFL) that dynamically adjusts class weights and focusing factors to curb head over-fitting and reduce popularity bias; (ii) Prototype Learning for Long-Tail Recommendation, which selects semantic, affective, and contextual prototypes to guide clustering and stabilize body and tail representations; and (iii) a GPT-4o-driven prototype-guided dialogue augmentation module that automatically generates diverse long-tail conversational snippets to alleviate tail sparsity and distribution shift. Together, these strategies enable LumiCRS to markedly improve recommendation accuracy, diversity, and fairness: on the REDIAL and INSPIRED benchmarks, LumiCRS boosts Recall@10 and Tail-Recall@10 by 7-15% over fifteen strong baselines, while human evaluations confirm superior fluency, informativeness, and long-tail relevance. These results demonstrate the effectiveness of multi-layer collaboration in building an efficient and fair long-tail conversational recommender.