All Grains, One Scheme (AGOS): Learning Multi-grain Instance Representation for Aerial Scene Classification

TL;DR

The paper introduces AGOS, a novel multi-grain learning framework for aerial scene classification that effectively captures multi-scale object information and fuses it for improved accuracy.

Contribution

It is the first to extend multiple instance learning into a multi-grain formulation for aerial scene classification, integrating multi-scale perception and semantic fusion modules.

Findings

Achieves competitive performance on UCM, AID, and NWPU benchmarks.

Effectively captures multi-scale object information in aerial images.

Demonstrates flexibility and easy integration with existing CNNs.

Abstract

Aerial scene classification remains challenging as: 1) the size of key objects in determining the scene scheme varies greatly; 2) many objects irrelevant to the scene scheme are often flooded in the image. Hence, how to effectively perceive the region of interests (RoIs) from a variety of sizes and build more discriminative representation from such complicated object distribution is vital to understand an aerial scene. In this paper, we propose a novel all grains, one scheme (AGOS) framework to tackle these challenges. To the best of our knowledge, it is the first work to extend the classic multiple instance learning into multi-grain formulation. Specially, it consists of a multi-grain perception module (MGP), a multi-branch multi-instance representation module (MBMIR) and a self-aligned semantic fusion (SSF) module. Firstly, our MGP preserves the differential dilated convolutional features from the backbone, which magnifies the discriminative information from multi-grains. Then, our MBMIR highlights the key instances in the multi-grain representation under the MIL formulation. Finally, our SSF allows our framework to learn the same scene scheme from multi-grain instance representations and fuses them, so that the entire framework is optimized as a whole. Notably, our AGOS is flexible and can be easily adapted to existing CNNs in a plug-and-play manner. Extensive experiments on UCM, AID and NWPU benchmarks demonstrate that our AGOS achieves a comparable performance against the state-of-the-art methods.