Improving Multi-label Recognition using Class Co-Occurrence Probabilities

TL;DR

This paper introduces a method that enhances multi-label recognition by integrating class co-occurrence probabilities using a Graph Convolutional Network, leading to improved performance over existing methods.

Contribution

The paper proposes a novel framework that incorporates class co-occurrence information via GCNs to refine classifiers in multi-label recognition tasks.

Findings

Outperforms state-of-the-art methods on four datasets.

Effectively captures class correlations to improve recognition accuracy.

Enhances vision-language model-based classifiers with co-occurrence data.

Abstract

Multi-label Recognition (MLR) involves the identification of multiple objects within an image. To address the additional complexity of this problem, recent works have leveraged information from vision-language models (VLMs) trained on large text-images datasets for the task. These methods learn an independent classifier for each object (class), overlooking correlations in their occurrences. Such co-occurrences can be captured from the training data as conditional probabilities between a pair of classes. We propose a framework to extend the independent classifiers by incorporating the co-occurrence information for object pairs to improve the performance of independent classifiers. We use a Graph Convolutional Network (GCN) to enforce the conditional probabilities between classes, by refining the initial estimates derived from image and text sources obtained using VLMs. We validate our method on four MLR datasets, where our approach outperforms all state-of-the-art methods.