HELM: Hierarchical and Explicit Label Modeling with Graph Learning for Multi-Label Image Classification

TL;DR

HELM introduces a hierarchical label modeling framework using Vision Transformers and graph convolutional networks, effectively capturing complex label dependencies and leveraging unlabeled data for improved multi-label image classification in remote sensing.

Contribution

The paper presents HELM, a novel approach combining hierarchy-specific class tokens, graph convolutional networks, and self-supervised learning to address multi-path hierarchies and unlabeled data in remote sensing image classification.

Findings

Achieves state-of-the-art results on four remote sensing datasets.

Outperforms baselines in both supervised and semi-supervised settings.

Excels particularly in low-label scenarios.

Abstract

Hierarchical multi-label classification (HMLC) is essential for modeling complex label dependencies in remote sensing. Existing methods, however, struggle with multi-path hierarchies where instances belong to multiple branches, and they rarely exploit unlabeled data. We introduce HELM (\textit{Hierarchical and Explicit Label Modeling}), a novel framework that overcomes these limitations. HELM: (i) uses hierarchy-specific class tokens within a Vision Transformer to capture nuanced label interactions; (ii) employs graph convolutional networks to explicitly encode the hierarchical structure and generate hierarchy-aware embeddings; and (iii) integrates a self-supervised branch to effectively leverage unlabeled imagery. We perform a comprehensive evaluation on four remote sensing image (RSI) datasets (UCM, AID, DFC-15, MLRSNet). HELM achieves state-of-the-art performance, consistently outperforming strong baselines in both supervised and semi-supervised settings, demonstrating particular strength in low-label scenarios.