On Learning Latent Models with Multi-Instance Weak Supervision

TL;DR

This paper introduces a theoretical framework for multi-instance Partial Label Learning (PLL) with possibly unknown transition functions, providing learnability conditions, error bounds, and empirical validation.

Contribution

It offers the first theoretical analysis of multi-instance PLL with unknown transitions, including learnability conditions and error bounds, extending existing PLL theory.

Findings

Learnability condition generalizes existing PLL results

Derived Rademacher-style error bounds for the problem

Empirical experiments validate theoretical insights and reveal scalability issues

Abstract

We consider a weakly supervised learning scenario where the supervision signal is generated by a transition function $\sigma$ of labels associated with multiple input instances. We formulate this problem as \emph{multi-instance Partial Label Learning (multi-instance PLL)}, which is an extension to the standard PLL problem. Our problem is met in different fields, including latent structural learning and neuro-symbolic integration. Despite the existence of many learning techniques, limited theoretical analysis has been dedicated to this problem. In this paper, we provide the first theoretical study of multi-instance PLL with possibly an unknown transition $\sigma$. Our main contributions are as follows. Firstly, we propose a necessary and sufficient condition for the learnability of the problem. This condition non-trivially generalizes and relaxes the existing small ambiguity degree in the PLL literature, since we allow the transition to be deterministic. Secondly, we derive Rademacher-style error bounds based on a top-$k$ surrogate loss that is widely used in the neuro-symbolic literature. Furthermore, we conclude with empirical experiments for learning under unknown transitions. The empirical results align with our theoretical findings; however, they also expose the issue of scalability in the weak supervision literature.