Addressing Correlated Latent Exogenous Variables in Debiased Recommender Systems

TL;DR

This paper introduces a novel learning algorithm for recommender systems that accounts for correlated latent exogenous variables, improving bias correction by modeling data generation with likelihood maximization.

Contribution

It relaxes the assumption of independence among exogenous variables in causal models, proposing a likelihood-based method to handle correlated latent variables in biased recommendation data.

Findings

Effective on synthetic datasets

Improves bias correction in real-world datasets

Demonstrates robustness to correlated exogenous variables

Abstract

Recommendation systems (RS) aim to provide personalized content, but they face a challenge in unbiased learning due to selection bias, where users only interact with items they prefer. This bias leads to a distorted representation of user preferences, which hinders the accuracy and fairness of recommendations. To address the issue, various methods such as error imputation based, inverse propensity scoring, and doubly robust techniques have been developed. Despite the progress, from the structural causal model perspective, previous debiasing methods in RS assume the independence of the exogenous variables. In this paper, we release this assumption and propose a learning algorithm based on likelihood maximization to learn a prediction model. We first discuss the correlation and difference between unmeasured confounding and our scenario, then we propose a unified method that effectively handles latent exogenous variables. Specifically, our method models the data generation process with latent exogenous variables under mild normality assumptions. We then develop a Monte Carlo algorithm to numerically estimate the likelihood function. Extensive experiments on synthetic datasets and three real-world datasets demonstrate the effectiveness of our proposed method. The code is at https://github.com/WallaceSUI/kdd25-background-variable.