PartImageNet++ Dataset: Scaling up Part-based Models for Robust Recognition

TL;DR

This paper introduces PIN++, a new large-scale dataset with high-quality part annotations for ImageNet-1K, and proposes a multi-scale part-supervised model that enhances adversarial robustness and generalization in object recognition.

Contribution

The paper provides the first large-scale part-annotated dataset for ImageNet-1K and introduces a novel multi-scale part-supervised model for robust recognition.

Findings

MPM improves adversarial robustness over strong baselines.

MPM shows increased robustness to common corruptions.

MPM generalizes well to out-of-distribution datasets.

Abstract

Deep learning-based object recognition systems can be easily fooled by various adversarial perturbations. One reason for the weak robustness may be that they do not have part-based inductive bias like the human recognition process. Motivated by this, several part-based recognition models have been proposed to improve the adversarial robustness of recognition. However, due to the lack of part annotations, the effectiveness of these methods is only validated on small-scale nonstandard datasets. In this work, we propose PIN++, short for PartImageNet++, a dataset providing high-quality part segmentation annotations for all categories of ImageNet-1K (IN-1K). With these annotations, we build part-based methods directly on the standard IN-1K dataset for robust recognition. Different from previous two-stage part-based models, we propose a Multi-scale Part-supervised Model (MPM), to learn a robust representation with part annotations. Experiments show that MPM yielded better adversarial robustness on the large-scale IN-1K over strong baselines across various attack settings. Furthermore, MPM achieved improved robustness on common corruptions and several out-of-distribution datasets. The dataset, together with these results, enables and encourages researchers to explore the potential of part-based models in more real applications.