Concept Matching with Agent for Out-of-Distribution Detection

TL;DR

This paper introduces Concept Matching with Agent (CMA), a novel method that enhances out-of-distribution detection in large language models by using agent-based prompts to improve robustness and accuracy.

Contribution

It integrates the agent paradigm into OOD detection, employing dynamic neutral prompts to form a triangle relationship for better separation of in-distribution and OOD data.

Findings

CMA outperforms existing methods in diverse real-world scenarios.

The triangular relationship improves ID and OOD separation.

CMA demonstrates robustness across various datasets.

Abstract

The remarkable achievements of Large Language Models (LLMs) have captivated the attention of both academia and industry, transcending their initial role in dialogue generation. To expand the usage scenarios of LLM, some works enhance the effectiveness and capabilities of the model by introducing more external information, which is called the agent paradigm. Based on this idea, we propose a new method that integrates the agent paradigm into out-of-distribution (OOD) detection task, aiming to improve its robustness and adaptability. Our proposed method, Concept Matching with Agent (CMA), employs neutral prompts as agents to augment the CLIP-based OOD detection process. These agents function as dynamic observers and communication hubs, interacting with both In-distribution (ID) labels and data inputs to form vector triangle relationships. This triangular framework offers a more nuanced approach than the traditional binary relationship, allowing for better separation and identification of ID and OOD inputs. Our extensive experimental results showcase the superior performance of CMA over both zero-shot and training-required methods in a diverse array of real-world scenarios.