TL;DR
This paper introduces PESO, a novel continual learning method for LLM-based recommender systems that balances adaptation to new user preferences with retention of recent behaviors using a proximal regularizer.
Contribution
PESO is the first to apply a proximal regularizer to LoRA adapters, improving continual adaptation in recommendation systems by better handling evolving user preferences.
Findings
PESO outperforms existing LoRA-based continual learning methods.
The proximal regularizer provides data-aware guidance in the LoRA subspace.
Theoretically, PESO offers a balanced approach to adaptation and preservation.
Abstract
While large language models (LLMs) achieve strong performance in recommendation, they face challenges in continual learning as users, items, and user preferences evolve over time. Existing LoRA-based continual methods primarily focus on preserving performance on previous tasks, but this overlooks the unique nature of recommendation: the goal is not to predict past preferences, and outdated preferences can even harm performance when current interests shift significantly. To address this, we propose PESO (Proximally rEgularized Single evolving lOra, a continual adaptation method for LoRA in recommendation. PESO introduces a proximal regularizer that anchors the current adapter to its most recent frozen state, enabling the model to flexibly balance adaptation and preservation, and to better capture recent user behaviors. Theoretically, we show that this proximal design provides data-aware,…
Peer Reviews
Decision·ICLR 2026 Poster
1. The paper tackles continual learning in recommendation, a fundamental and practical problem. 2. The motivation is clear, with comprehensive analysis of single vs. cumulative LoRA and their limitations in the recommendation setting. 3. The theoretical justification for the proximal regularization providing data-aware, direction-wise guidance in the LoRA subspace is well presented. 4. Experiments are systematic and demonstrate consistent gains over baselines.
1. The motivation for using LoRA as the primary PEFT technique is not fully convincing. Comparison or discussion with alternatives such as prompt tuning or layer pruning is missing. 2. The paper lacks discussion and comparison with recent works on continual or incremental learning for LLM-based generative recommendation [1]. 3. The experimental datasets lack diversity, all drawn from the Amazon Review corpus in the e-commerce domain, limiting the generalization of conclusions. 4. While the th
1. The paper is well-written and easy to follow, with a solid theoretical analysis for the proposed PESO. 2. The choice of a modern, semantic ID-based generative recommendation as the experimental backbone is highly relevant and makes the results more convincing.
1. A key weakness is the practical relevance of the problem formulation. The paper partitions data into discrete, chronological blocks to simulate a continual learning scenario. However, in real-world RS that are updated frequently (often in near real-time), the data distribution shift between consecutive updates is typically small and gradual. The paper fails to quantify the distribution shift in its experimental data splits, making it unclear if the problem it solves reflects realistic deploym
1. **Novel problem framing:** The paper clearly articulates how continual recommendation differs from general continual learning, emphasizing the role of evolving user preferences instead of task retention. This perspective grounds the LLM-based recommendersa in realistic recommender settings. 2. **Simple but Effective:** PESO avoids multiple adapters (reducing storage and interference) and introduces a lightweight proximal term that yields theoretically grounded, data-aware stability—achieving
1. **Limited modeling of long-term preference dynamics：** PESO is evaluated on short-term chronological splits (four-stage Amazon data). How would it behave under nonlinear or cyclical preference drifts over long horizons? Would the single-step proximal constraint still be sufficient, or would multi-timescale or memory-based mechanisms be required? 2. **Unanalyzed efficiency trade-offs.** PESO maintains previous adapter states for proximal computation. What is the actual storage and computation
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
