Orderless Recurrent Models for Multi-label Classification

TL;DR

This paper introduces a dynamic label ordering method for RNN-based multi-label classification that adapts to each image, leading to faster training and improved accuracy over traditional fixed ordering approaches.

Contribution

It proposes a novel dynamic label ordering technique for RNNs in multi-label classification, enhancing training efficiency and model performance.

Findings

Outperforms existing CNN-RNN models on MS-COCO, WIDER Attribute, and PA-100K datasets.

Avoids duplicate label generation common in other models.

Achieves state-of-the-art results with a standard encoder-decoder architecture.

Abstract

Recurrent neural networks (RNN) are popular for many computer vision tasks, including multi-label classification. Since RNNs produce sequential outputs, labels need to be ordered for the multi-label classification task. Current approaches sort labels according to their frequency, typically ordering them in either rare-first or frequent-first. These imposed orderings do not take into account that the natural order to generate the labels can change for each image, e.g.\ first the dominant object before summing up the smaller objects in the image. Therefore, in this paper, we propose ways to dynamically order the ground truth labels with the predicted label sequence. This allows for the faster training of more optimal LSTM models for multi-label classification. Analysis evidences that our method does not suffer from duplicate generation, something which is common for other models. Furthermore, it outperforms other CNN-RNN models, and we show that a standard architecture of an image encoder and language decoder trained with our proposed loss obtains the state-of-the-art results on the challenging MS-COCO, WIDER Attribute and PA-100K and competitive results on NUS-WIDE.