Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning

TL;DR

This paper introduces GEMS, a parameter-efficient fine-tuning framework that unifies search and recommendation tasks in LLMs by reducing gradient conflicts and preserving general knowledge, leading to improved performance.

Contribution

GEMS proposes a novel multi-subspace decomposition and null-space projection method to effectively unify search and recommendation in LLMs while maintaining scalability and knowledge integrity.

Findings

GEMS outperforms state-of-the-art baselines on benchmark datasets.

It effectively reduces gradient conflicts across tasks.

It preserves general-domain knowledge during fine-tuning.

Abstract

Search and recommendation (S&R) are core to online platforms, addressing explicit intent through queries and modeling implicit intent from behaviors, respectively. Their complementary roles motivate a unified modeling paradigm. Early studies to unify S&R adopt shared encoders with task-specific heads, while recent efforts reframe item ranking in both S&R as conditional generation. The latter holds particular promise, enabling end-to-end optimization and leveraging the semantic understanding of LLMs. However, existing methods rely on full fine-tuning, which is computationally expensive and limits scalability. Parameter-efficient fine-tuning (PEFT) offers a more practical alternative but faces two critical challenges in unifying S&R: (1) gradient conflicts across tasks due to divergent optimization objectives, and (2) shifts in user intent understanding caused by overfitting to fine-tuning data, which distort general-domain knowledge and weaken LLM reasoning. To address the above issues, we propose Gradient Multi-Subspace Tuning (GEMS), a novel framework that unifies S&R with LLMs while alleviating gradient conflicts and preserving general-domain knowledge. GEMS introduces (1) \textbf{Multi-Subspace Decomposition}, which disentangles shared and task-specific optimization signals into complementary low-rank subspaces, thereby reducing destructive gradient interference, and (2) \textbf{Null-Space Projection}, which constrains parameter updates to a subspace orthogonal to the general-domain knowledge space, mitigating shifts in user intent understanding. Extensive experiments on benchmark datasets show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks, achieving superior effectiveness.