Decorrelating Structure via Adapters Makes Ensemble Learning Practical for Semi-supervised Learning

TL;DR

This paper introduces DSA, a lightweight, architecture-agnostic ensemble method that decorrelates prediction heads to improve deep network reliability and robustness across various visual tasks, without complex regularization.

Contribution

The paper proposes DSA, a novel decorrelating adapter structure that enhances ensemble learning efficiency and robustness without additional loss functions or regularization.

Findings

Achieved 5.35% accuracy improvement on CIFAR-10 with 40 labels.

Improved PCK by up to 5.2% on multiple keypoint datasets.

Lower bias and variance compared to single-head methods.

Abstract

In computer vision, traditional ensemble learning methods exhibit either a low training efficiency or the limited performance to enhance the reliability of deep neural networks. In this paper, we propose a lightweight, loss-function-free, and architecture-agnostic ensemble learning by the Decorrelating Structure via Adapters (DSA) for various visual tasks. Concretely, the proposed DSA leverages the structure-diverse adapters to decorrelate multiple prediction heads without any tailed regularization or loss. This allows DSA to be easily extensible to architecture-agnostic networks for a range of computer vision tasks. Importantly, the theoretically analysis shows that the proposed DSA has a lower bias and variance than that of the single head based method (which is adopted by most of the state of art approaches). Consequently, the DSA makes deep networks reliable and robust for the various real-world challenges, \textit{e.g.}, data corruption, and label noises. Extensive experiments combining the proposed method with FreeMatch achieved the accuracy improvements of 5.35% on CIFAR-10 dataset with 40 labeled data and 0.71% on CIFAR-100 dataset with 400 labeled data. Besides, combining the proposed method with DualPose achieved the improvements in the Percentage of Correct Keypoints (PCK) by 2.08% on the Sniffing dataset with 100 data (30 labeled data), 5.2% on the FLIC dataset with 100 data (including 50 labeled data), and 2.35% on the LSP dataset with 200 data (100 labeled data).