Unified Risk Analysis for Weakly Supervised Learning

TL;DR

This paper introduces a unified framework for understanding and systematically addressing risk rewrite problems in weakly supervised learning, encompassing various settings through contamination interpretation and novel decontamination strategies.

Contribution

It provides a comprehensive interpretation of weak supervision mechanisms and proposes a new risk rewrite method called marginal chain, unifying existing approaches.

Findings

Framework subsumes 15 WSL settings

Introduces marginal chain decontamination strategy

Recovers existing risk rewrite methods

Abstract

Among the flourishing research of weakly supervised learning (WSL), we recognize the lack of a unified interpretation of the mechanism behind the weakly supervised scenarios, let alone a systematic treatment of the risk rewrite problem, a crucial step in the empirical risk minimization approach. In this paper, we introduce a framework providing a comprehensive understanding and a unified methodology for WSL. The formulation component of the framework, leveraging a contamination perspective, provides a unified interpretation of how weak supervision is formed and subsumes fifteen existing WSL settings. The induced reduction graphs offer comprehensive connections over WSLs. The analysis component of the framework, viewed as a decontamination process, provides a systematic method of conducting risk rewrite. In addition to the conventional inverse matrix approach, we devise a novel strategy called marginal chain aiming to decontaminate distributions. We justify the feasibility of the proposed framework by recovering existing rewrites reported in the literature.