Out-of-Distribution Learning with Human Feedback

TL;DR

This paper introduces a novel framework for out-of-distribution learning that leverages human feedback and unlabeled wild data to improve model robustness and detection capabilities in real-world scenarios.

Contribution

It proposes a new approach combining human feedback with selective labeling of wild data to enhance OOD generalization and detection, supported by theoretical analysis and extensive experiments.

Findings

Outperforms current state-of-the-art methods significantly.

Provides theoretical bounds on generalization error.

Effectively utilizes unlabeled wild data with minimal human labeling.

Abstract

Out-of-distribution (OOD) learning often relies heavily on statistical approaches or predefined assumptions about OOD data distributions, hindering their efficacy in addressing multifaceted challenges of OOD generalization and OOD detection in real-world deployment environments. This paper presents a novel framework for OOD learning with human feedback, which can provide invaluable insights into the nature of OOD shifts and guide effective model adaptation. Our framework capitalizes on the freely available unlabeled data in the wild that captures the environmental test-time OOD distributions under both covariate and semantic shifts. To harness such data, our key idea is to selectively provide human feedback and label a small number of informative samples from the wild data distribution, which are then used to train a multi-class classifier and an OOD detector. By exploiting human feedback, we enhance the robustness and reliability of machine learning models, equipping them with the capability to handle OOD scenarios with greater precision. We provide theoretical insights on the generalization error bounds to justify our algorithm. Extensive experiments show the superiority of our method, outperforming the current state-of-the-art by a significant margin.