Towards Non-I.I.D. Image Classification: A Dataset and Baselines

TL;DR

This paper introduces NICO, a new Non-I.I.D. image dataset, and baseline models to study and improve image classification performance when training and testing data distributions differ.

Contribution

The paper creates and releases NICO, a flexible Non-I.I.D. dataset, and proposes baseline ConvNet models with batch balancing for Non-I.I.D. image classification.

Findings

NICO supports various Non-I.I.D. scenarios effectively.

ConvNet models trained on NICO perform well in Non-I.I.D. settings.

Batch balancing improves ConvNet performance in Non-I.I.D. conditions.

Abstract

I.I.D. hypothesis between training and testing data is the basis of numerous image classification methods. Such property can hardly be guaranteed in practice where the Non-IIDness is common, causing instable performances of these models. In literature, however, the Non-I.I.D. image classification problem is largely understudied. A key reason is lacking of a well-designed dataset to support related research. In this paper, we construct and release a Non-I.I.D. image dataset called NICO, which uses contexts to create Non-IIDness consciously. Compared to other datasets, extended analyses prove NICO can support various Non-I.I.D. situations with sufficient flexibility. Meanwhile, we propose a baseline model with ConvNet structure for General Non-I.I.D. image classification, where distribution of testing data is unknown but different from training data. The experimental results demonstrate that NICO can well support the training of ConvNet model from scratch, and a batch balancing module can help ConvNets to perform better in Non-I.I.D. settings.