FastBUS: A Fast Bayesian Framework for Unified Weakly-Supervised Learning

TL;DR

FastBUS introduces a unified Bayesian framework that efficiently handles various weakly supervised learning tasks, significantly reducing computation time while maintaining state-of-the-art performance.

Contribution

The paper presents a novel probabilistic approach that compresses diverse weak supervision structures into a shared Bayesian network, enabling faster and more scalable inference.

Findings

Achieves state-of-the-art results across multiple weak supervision scenarios.

Provides up to hundreds of times faster inference compared to existing methods.

Demonstrates the equivalence of the proposed method with the EM algorithm in most cases.

Abstract

Machine Learning often involves various imprecise labels, leading to diverse weakly supervised settings. While recent methods aim for universal handling, they usually suffer from complex manual pre-work, ignore the relationships between associated labels, or are unable to batch process due to computational design flaws, resulting in long running times. To address these limitations, we propose a novel general framework that efficiently infers latent true label distributions across various weak supervisions. Our key idea is to express the label brute-force search process as a probabilistic transition of label variables, compressing diverse weakly supervised DFS tree structures into a shared Bayesian network. From this, we derived a latent probability calculation algorithm based on generalized belief propagation and proposed two joint acceleration strategies: 1) introducing a low-rank assumption to approximate the transition matrix, reducing time complexity; 2) designing an end-to-end state evolution module to learn batch-scale transition matrices, facilitating multi-category batch processing. In addition, the equivalence of our method with the EM algorithm in most scenarios is further demonstrated. Extensive experiments show that our method achieves SOTA results under most weakly supervised settings, and achieves up to hundreds of times faster acceleration in running time compared to other general methods.