Benchmarking In-context Experiential Learning Through Repeated Product Recommendations

TL;DR

This paper introduces BELA, a benchmark for evaluating in-context experiential learning in product recommendation tasks, highlighting current models' limitations in adaptive learning through dialogue.

Contribution

The paper presents a new benchmark, BELA, combining real-world data, diverse personas, and a simulated environment to assess experiential learning in language models.

Findings

Current models show limited improvement across episodes.

BELA enables evaluation of adaptive learning in dialogue-based recommendation.

Highlighting the need for models with stronger in-context learning capabilities.

Abstract

To reliably navigate ever-shifting real-world environments, agents must grapple with incomplete knowledge and adapt their behavior through experience. However, current evaluations largely focus on tasks that leave no ambiguity, and do not measure agents' ability to adaptively learn and reason through the experiences they accrued. We exemplify the need for this in-context experiential learning in a product recommendation context, where agents must navigate shifting customer preferences and product landscapes through natural language dialogue. We curate a benchmark for experiential learning and active exploration (BELA) that combines (1) rich real-world products from Amazon, (2) a diverse collection of user personas to represent heterogeneous yet latent preferences, and (3) a LLM user simulator powered by the persona to create rich interactive trajectories. We observe that current frontier models struggle to meaningfully improve across episodes, underscoring the need for agentic systems with strong in-context learning capabilities.