Be Aware of the Neighborhood Effect: Modeling Selection Bias under Interference

TL;DR

This paper addresses selection bias in recommender systems by modeling neighborhood effects as interference, proposing a novel loss and estimators to achieve unbiased learning in the presence of both bias and interference.

Contribution

It introduces a formal causal inference framework for neighborhood effects, proposing a new ideal loss and estimators that account for interference in bias correction.

Findings

Proposed methods achieve unbiased learning with neighborhood effects.

Theoretical connection established between new and existing debiasing methods.

Experimental results demonstrate effectiveness on real-world data.

Abstract

Selection bias in recommender system arises from the recommendation process of system filtering and the interactive process of user selection. Many previous studies have focused on addressing selection bias to achieve unbiased learning of the prediction model, but ignore the fact that potential outcomes for a given user-item pair may vary with the treatments assigned to other user-item pairs, named neighborhood effect. To fill the gap, this paper formally formulates the neighborhood effect as an interference problem from the perspective of causal inference and introduces a treatment representation to capture the neighborhood effect. On this basis, we propose a novel ideal loss that can be used to deal with selection bias in the presence of neighborhood effect. We further develop two new estimators for estimating the proposed ideal loss. We theoretically establish the connection between the proposed and previous debiasing methods ignoring the neighborhood effect, showing that the proposed methods can achieve unbiased learning when both selection bias and neighborhood effect are present, while the existing methods are biased. Extensive semi-synthetic and real-world experiments are conducted to demonstrate the effectiveness of the proposed methods.