SR-OOD: Out-of-Distribution Detection via Sample Repairing
Rui Sun, Andi Zhang, Haiming Zhang, Jinke Ren, Yao Zhu, Ruimao Zhang,, Shuguang Cui, Zhen Li
TL;DR
SR-OOD introduces a novel out-of-distribution detection method that repairs samples using a generative adversarial network to identify semantic inconsistencies without needing extra data or labels.
Contribution
The paper proposes a new OOD detection framework leveraging sample repairing and distance metrics, outperforming existing generative methods without requiring additional data.
Findings
Achieves superior OOD detection performance on CIFAR-10, CelebA, and Pokemon datasets.
Does not require extra data or labels for detection.
Outperforms state-of-the-art generative methods.
Abstract
Out-of-distribution (OOD) detection is a crucial task for ensuring the reliability and robustness of machine learning models. Recent works have shown that generative models often assign high confidence scores to OOD samples, indicating that they fail to capture the semantic information of the data. To tackle this problem, we take advantage of sample repairing and propose a novel OOD detection framework, namely SR-OOD. Our framework leverages the idea that repairing an OOD sample can reveal its semantic inconsistency with the in-distribution data. Specifically, our framework consists of two components: a sample repairing module and a detection module. The sample repairing module applies erosion to an input sample and uses a generative adversarial network to repair it. The detection module then determines whether the input sample is OOD using a distance metric. Our framework does not…
Peer Reviews
Decision·ICLR 2024 Conference Withdrawn Submission
- The idea of adding an extra perception network for original and reconstruction samples intuitively makes sense. - There are different kinds of downgrading transformation e.g, masking, super-resolution, that have been explored for image repairing.
Novelty: - My major concern is that this paper explored a very similar idea to an ECCV 2022 paper [1]. This paper also proposes to repair a masked image with a reconstruction network and then use a further perception network to compute the OOD score for semantic inconsistency. The minor difference from my view is that this paper explored super-resolution as a new type of downsampling augmentation. But overall, they are very similar in high-level ideas. Experiment: - I am wondering why authors
1. A framework for OOD detection based on sample repair is proposed, which is a novel and effective idea. 2. The framework in this paper does not require additional data or labeling information or time-consuming processes, so it can be applied to a variety of scenarios and datasets. 3. The authors provide some diagrams and pictures to show the effect and principle of their approach, with detailed derivation, principle introduction and rich experiments.
1. Not enough details and formulas are given to show how they implement sample repair and OOD detection. For example, in Section 3.1, the authors do not give specifics on how to select the erosion operation T and how to compute the distance metric S(x). In Section 3.2, the authors do not give a specific procedure on how to select the style mixing parameters and how to evaluate the different erosion methods T∗ . 2. In the experimental section, the authors do not analyze and discuss the experiment
The idea is in general intuitive and easy-to-follow.
### Ambiguity in (theoretical) analysis 1. While the authors clearly state that the hypotheses and claims in Sec. 2 are not formal (and I understand it), I still find many of them are too loose such that clarity is significantly harmed. For example, Claim 1 states that "The OOD data ... should have distinct shared features compared to the in-distribution data." However, in the meantime Claim 2 says "The shared features of the SVHN dataset have a significant overlap with those of the CIFAR-10 dat
1. The method intuitively makes sense for far-OOD datasets. In general, I agree with the authors on the hypothesis and claims for far-OOD datasets. The ID and far-OOD datasets can be significantly different so repairing can amplify the semantic inconsistency.
1. **Usefulness of near-OOD datasets.** As I wrote above, the methodology should work for far-OOD datasets, but how about the near-OOD datasets such as FashionMNIST versus MNIST and CIFAR10 versus CIFAR100? I doubt the efficacy of the methodology for near-OOD datasets as the repairing can be similar. 2. **Scalability to large-scale datasets.** Another concern is that when the training ID dataset scales up to large-scale datasets such as ImageNet-1k, how would the network repair eroded samples
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAnomaly Detection Techniques and Applications · Generative Adversarial Networks and Image Synthesis · Adversarial Robustness in Machine Learning
MethodsFocus