Towards Confidence-aware Calibrated Recommendation

TL;DR

This paper introduces a confidence-aware re-ranking algorithm for recommender systems that balances calibration, relevance, and diversity, improving performance by considering calibration confidence based on user profile size.

Contribution

It proposes a novel optimization-based re-ranking method that incorporates calibration confidence, addressing limitations of existing calibration techniques in recommender systems.

Findings

Outperforms state-of-the-art methods in accuracy metrics

Balances calibration, relevance, and diversity effectively

Considers calibration confidence based on user profile size

Abstract

Recommender systems utilize users' historical data to learn and predict their future interests, providing them with suggestions tailored to their tastes. Calibration ensures that the distribution of recommended item categories is consistent with the user's historical data. Mitigating miscalibration brings various benefits to a recommender system. For example, it becomes less likely that a system overlooks categories with less interaction on a user's profile by only recommending popular categories. Despite the notable success, calibration methods have several drawbacks, such as limiting the diversity of the recommended items and not considering the calibration confidence. This work, presents a set of properties that address various aspects of a desired calibrated recommender system. Considering these properties, we propose a confidence-aware optimization-based re-ranking algorithm to find the balance between calibration, relevance, and item diversity, while simultaneously accounting for calibration confidence based on user profile size. Our model outperforms state-of-the-art methods in terms of various accuracy and beyond-accuracy metrics for different user groups.