Exploration on Demand: From Algorithmic Control to User Empowerment

TL;DR

This paper presents an adaptive clustering framework with user-controlled exploration for recommender systems, balancing personalization and diversity to reduce filter bubbles and enhance content discovery.

Contribution

It introduces a novel clustering and exploration mechanism that allows users to control diversity, improving recommendation variety without losing relevance.

Findings

Exploration reduces intra-list similarity from 0.34 to 0.26

Unexpectedness increases to 0.73 with exploration

72.7% of simulated users prefer exploratory recommendations

Abstract

Recommender systems often struggle with over-specialization, which severely limits users' exposure to diverse content and creates filter bubbles that reduce serendipitous discovery. To address this fundamental limitation, this paper introduces an adaptive clustering framework with user-controlled exploration that effectively balances personalization and diversity in movie recommendations. Our approach leverages sentence-transformer embeddings to group items into semantically coherent clusters through an online algorithm with dynamic thresholding, thereby creating a structured representation of the content space. Building upon this clustering foundation, we propose a novel exploration mechanism that empowers users to control recommendation diversity by strategically sampling from less-engaged clusters, thus expanding their content horizons while preserving relevance. Experiments on the MovieLens dataset demonstrate the system's effectiveness, showing that exploration significantly reduces intra-list similarity from 0.34 to 0.26 while simultaneously increasing unexpectedness to 0.73. Furthermore, our Large Language Model-based A/B testing methodology, conducted with 300 simulated users, reveals that 72.7% of long-term users prefer exploratory recommendations over purely exploitative ones, providing strong evidence for the system's ability to promote meaningful content discovery without sacrificing user satisfaction.