Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detection

TL;DR

This paper introduces Local-Prompt, a novel approach for out-of-distribution detection that emphasizes local information through a coarse-to-fine tuning paradigm, significantly improving detection performance in few-shot scenarios.

Contribution

The paper proposes a new Local-Prompt method that utilizes local prompts and regional regularization, enhancing OOD detection by leveraging local outlier knowledge while maintaining compatibility with global prompts.

Findings

Reduces average FPR95 by 5.17% in 4-shot setting on ImageNet-1k.

Outperforms state-of-the-art methods even with fewer shots.

Effectively integrates local prompts with global prompts during inference.

Abstract

Out-of-Distribution (OOD) detection, aiming to distinguish outliers from known categories, has gained prominence in practical scenarios. Recently, the advent of vision-language models (VLM) has heightened interest in enhancing OOD detection for VLM through few-shot tuning. However, existing methods mainly focus on optimizing global prompts, ignoring refined utilization of local information with regard to outliers. Motivated by this, we freeze global prompts and introduce Local-Prompt, a novel coarse-to-fine tuning paradigm to emphasize regional enhancement with local prompts. Our method comprises two integral components: global prompt guided negative augmentation and local prompt enhanced regional regularization. The former utilizes frozen, coarse global prompts as guiding cues to incorporate negative augmentation, thereby leveraging local outlier knowledge. The latter employs trainable local prompts and a regional regularization to capture local information effectively, aiding in outlier identification. We also propose regional-related metric to empower the enrichment of OOD detection. Moreover, since our approach explores enhancing local prompts only, it can be seamlessly integrated with trained global prompts during inference to boost the performance. Comprehensive experiments demonstrate the effectiveness and potential of our method. Notably, our method reduces average FPR95 by 5.17% against state-of-the-art method in 4-shot tuning on challenging ImageNet-1k dataset, even outperforming 16-shot results of previous methods. Code is released at https://github.com/AuroraZengfh/Local-Prompt.