Towards Privacy-Preserving Fine-Grained Visual Classification via Hierarchical Learning from Label Proportions

TL;DR

This paper introduces a hierarchical learning framework that enables privacy-preserving fine-grained visual classification using only bag-level labels, improving accuracy by exploiting dataset hierarchy.

Contribution

It proposes LHFGLP, a novel hierarchical LLP framework with a hierarchical loss, transforming handcrafted methods into learnable models for privacy-sensitive fine-grained recognition.

Findings

Outperforms existing LLP-based methods on multiple datasets

Effectively leverages hierarchical dataset structure for better accuracy

Provides a learnable optimization approach for label proportion learning

Abstract

In recent years, Fine-Grained Visual Classification (FGVC) has achieved impressive recognition accuracy, despite minimal inter-class variations. However, existing methods heavily rely on instance-level labels, making them impractical in privacy-sensitive scenarios such as medical image analysis. This paper aims to enable accurate fine-grained recognition without direct access to instance labels. To achieve this, we leverage the Learning from Label Proportions (LLP) paradigm, which requires only bag-level labels for efficient training. Unlike existing LLP-based methods, our framework explicitly exploits the hierarchical nature of fine-grained datasets, enabling progressive feature granularity refinement and improving classification accuracy. We propose Learning from Hierarchical Fine-Grained Label Proportions (LHFGLP), a framework that incorporates Unrolled Hierarchical Fine-Grained Sparse Dictionary Learning, transforming handcrafted iterative approximation into learnable network optimization. Additionally, our proposed Hierarchical Proportion Loss provides hierarchical supervision, further enhancing classification performance. Experiments on three widely-used fine-grained datasets, structured in a bag-based manner, demonstrate that our framework consistently outperforms existing LLP-based methods. We will release our code and datasets to foster further research in privacy-preserving fine-grained classification.