Multi-Instance Causal Representation Learning for Instance Label Prediction and Out-of-Distribution Generalization

TL;DR

This paper introduces CausalMIL, a novel method that leverages multi-instance bags as auxiliary information to identify causal representations at the instance level, improving label prediction and robustness to distribution shifts.

Contribution

It proposes a new causal MIL framework using identifiable variational autoencoders, enhancing instance label prediction and out-of-distribution generalization.

Findings

Outperforms baseline methods on synthetic datasets.

Achieves superior instance label prediction accuracy.

Demonstrates robustness to distribution changes.

Abstract

Multi-instance learning (MIL) deals with objects represented as bags of instances and can predict instance labels from bag-level supervision. However, significant performance gaps exist between instance-level MIL algorithms and supervised learners since the instance labels are unavailable in MIL. Most existing MIL algorithms tackle the problem by treating multi-instance bags as harmful ambiguities and predicting instance labels by reducing the supervision inexactness. This work studies MIL from a new perspective by considering bags as auxiliary information, and utilize it to identify instance-level causal representations from bag-level weak supervision. We propose the CausalMIL algorithm, which not only excels at instance label prediction but also provides robustness to distribution change by synergistically integrating MIL with identifiable variational autoencoder. Our approach is based on a practical and general assumption: the prior distribution over the instance latent representations belongs to the non-factorized exponential family conditioning on the multi-instance bags. Experiments on synthetic and real-world datasets demonstrate that our approach significantly outperforms various baselines on instance label prediction and out-of-distribution generalization tasks.