Opening Deep Neural Networks with Generative Models

TL;DR

This paper introduces GeMOS, a simple and effective plug-and-play framework that combines pretrained neural networks with generative models to improve open set recognition, enabling better detection of unknown classes in visual tasks.

Contribution

The paper presents GeMOS, a novel, easy-to-integrate method that enhances open set recognition by pairing pretrained CNNs with generative models, outperforming or matching complex existing methods.

Findings

GeMOS outperforms state-of-the-art open set algorithms in evaluations.

GeMOS is simple, plug-and-play, and computationally efficient.

It effectively detects unknown classes in visual recognition tasks.

Abstract

Image classification methods are usually trained to perform predictions taking into account a predefined group of known classes. Real-world problems, however, may not allow for a full knowledge of the input and label spaces, making failures in recognition a hazard to deep visual learning. Open set recognition methods are characterized by the ability to correctly identify inputs of known and unknown classes. In this context, we propose GeMOS: simple and plug-and-play open set recognition modules that can be attached to pretrained Deep Neural Networks for visual recognition. The GeMOS framework pairs pre-trained Convolutional Neural Networks with generative models for open set recognition to extract open set scores for each sample, allowing for failure recognition in object recognition tasks. We conduct a thorough evaluation of the proposed method in comparison with state-of-the-art open set algorithms, finding that GeMOS either outperforms or is statistically indistinguishable from more complex and costly models.