Interaction-Data-guided Conditional Instrumental Variables for Debiasing Recommender Systems

TL;DR

This paper introduces IDCIV-RS, a novel method that automatically generates valid conditional instrumental variables from interaction data to reduce confounding bias in recommender systems, improving prediction accuracy.

Contribution

The proposed IDCIV-RS method automatically learns valid CIV representations and conditioning sets directly from interaction data using a variational autoencoder, simplifying IV selection and debiasing recommender systems.

Findings

IDCIV-RS effectively reduces bias in recommendations.

The method improves accuracy on real-world datasets.

It automates CIV generation from interaction data.

Abstract

It is often challenging to identify a valid instrumental variable (IV), although the IV methods have been regarded as effective tools of addressing the confounding bias introduced by latent variables. To deal with this issue, an Interaction-Data-guided Conditional IV (IDCIV) debiasing method is proposed for Recommender Systems, called IDCIV-RS. The IDCIV-RS automatically generates the representations of valid CIVs and their corresponding conditioning sets directly from interaction data, significantly reducing the complexity of IV selection while effectively mitigating the confounding bias caused by latent variables in recommender systems. Specifically, the IDCIV-RS leverages a variational autoencoder (VAE) to learn both the CIV representations and their conditioning sets from interaction data, followed by the application of least squares to derive causal representations for click prediction. Extensive experiments on two real-world datasets, Movielens-10M and Douban-Movie, demonstrate that IDCIV-RS successfully learns the representations of valid CIVs, effectively reduces bias, and consequently improves recommendation accuracy.