Class-Specific Semantic Reconstruction for Open Set Recognition

TL;DR

This paper introduces CSSR, a novel open set recognition method that uses class-specific auto-encoder manifolds to improve detection of unknown classes while maintaining accuracy on known classes.

Contribution

CSSR integrates class-specific auto-encoders with prototype learning, modeling each class on an individual AE manifold for better open set recognition.

Findings

Achieves outstanding performance on multiple datasets.

Effectively detects unknown classes while maintaining known class accuracy.

Simple and flexible to incorporate into existing frameworks.

Abstract

Open set recognition enables deep neural networks (DNNs) to identify samples of unknown classes, while maintaining high classification accuracy on samples of known classes. Existing methods basing on auto-encoder (AE) and prototype learning show great potential in handling this challenging task. In this study, we propose a novel method, called Class-Specific Semantic Reconstruction (CSSR), that integrates the power of AE and prototype learning. Specifically, CSSR replaces prototype points with manifolds represented by class-specific AEs. Unlike conventional prototype-based methods, CSSR models each known class on an individual AE manifold, and measures class belongingness through AE's reconstruction error. Class-specific AEs are plugged into the top of the DNN backbone and reconstruct the semantic representations learned by the DNN instead of the raw image. Through end-to-end learning, the DNN and the AEs boost each other to learn both discriminative and representative information. The results of experiments conducted on multiple datasets show that the proposed method achieves outstanding performance in both close and open set recognition and is sufficiently simple and flexible to incorporate into existing frameworks.