Beyond a Single Perspective: Text Anomaly Detection with Multi-View Language Representations

TL;DR

This paper introduces MCA^2, a multi-view framework for text anomaly detection that leverages multiple pretrained language models and adaptive modules to improve detection across diverse datasets and anomaly types.

Contribution

It proposes a novel multi-view TAD framework that integrates multiple embeddings, uses contrastive collaboration, and adaptively weights views for enhanced performance.

Findings

Outperforms strong baselines on 10 benchmark datasets.

Effectively captures normal textual patterns from multiple views.

Demonstrates adaptability across diverse datasets and anomaly types.

Abstract

Text anomaly detection (TAD) plays a critical role in various language-driven real-world applications, including harmful content moderation, phishing detection, and spam review filtering. While two-step "embedding-detector" TAD methods have shown state-of-the-art performance, their effectiveness is often limited by the use of a single embedding model and the lack of adaptability across diverse datasets and anomaly types. To address these limitations, we propose to exploit the embeddings from multiple pretrained language models and integrate them into $MCA^2$, a multi-view TAD framework. $MCA^2$ adopts a multi-view reconstruction model to effectively extract normal textual patterns from multiple embedding perspectives. To exploit inter-view complementarity, a contrastive collaboration module is designed to leverage and strengthen the interactions across different views. Moreover, an adaptive allocation module is developed to automatically assign the contribution weight of each view, thereby improving the adaptability to diverse datasets. Extensive experiments on 10 benchmark datasets verify the effectiveness of $MCA^2$ against strong baselines. The source code of $MCA^2$ is available at https://github.com/yankehan/MCA2.