Seeing the Needle in the Haystack: Towards Weakly-Supervised Log Instance Anomaly Localization via Counterfactual Perturbation

TL;DR

This paper introduces LogMILP, a weakly supervised framework for anomaly detection and localization in logs, using only bag-level labels and prototype-guided modeling with counterfactual perturbation, achieving reliable results.

Contribution

The paper presents LogMILP, a novel weakly supervised method that improves log anomaly localization accuracy and interpretability with minimal supervision.

Findings

Achieves competitive detection performance on public datasets.

Provides significantly more reliable instance-level localization.

Utilizes prototype-guided structural modeling with perturbation regularization.

Abstract

Log anomaly detection is a critical task for system operations and security assurance. However, in networked systems at scale, log data are generated at massive scale while instance-level annotations are prohibitively expensive, posing great difficulties to fine-grained anomaly localization. To address this challenge, we propose LogMILP (Log anomaly localization based on Multi-Instance Learning enhanced by prototypes and Perturbation), a weakly supervised framework that enables both bag-level anomaly detection and instance-level anomaly localization using only bag-level labels. Our method guides the model to pinpoint the critical log entries using prototype-guided structural modeling with counterfactual perturbation consistency regularization, thereby improving localization reliability and interpretability under coarse-grained supervision. Experimental results on three public datasets demonstrate that LogMILP achieves competitive detection performance while yielding significantly more reliable instance-level localization. Our code is open-sourced at https://github.com/YUK1207/LogMILP.