Few-Shot Hyperspectral Image Classification With Unknown Classes Using Multitask Deep Learning

TL;DR

This paper introduces a multitask deep learning approach for hyperspectral image classification that detects unknown classes by combining classification and reconstruction, improving accuracy in open-world scenarios with few labeled samples.

Contribution

The proposed MDL4OW method uniquely integrates classification and reconstruction to identify unknown classes in hyperspectral images, addressing the open-world challenge.

Findings

Achieved state-of-the-art accuracy on real-world hyperspectral datasets.

Improved overall accuracy by 4.94% on Salinas data.

Effectively detects unknown classes in few-shot scenarios.

Abstract

Current hyperspectral image classification assumes that a predefined classification system is closed and complete, and there are no unknown or novel classes in the unseen data. However, this assumption may be too strict for the real world. Often, novel classes are overlooked when the classification system is constructed. The closed nature forces a model to assign a label given a new sample and may lead to overestimation of known land covers (e.g., crop area). To tackle this issue, we propose a multitask deep learning method that simultaneously conducts classification and reconstruction in the open world (named MDL4OW) where unknown classes may exist. The reconstructed data are compared with the original data; those failing to be reconstructed are considered unknown, based on the assumption that they are not well represented in the latent features due to the lack of labels. A threshold needs to be defined to separate the unknown and known classes; we propose two strategies based on the extreme value theory for few-shot and many-shot scenarios. The proposed method was tested on real-world hyperspectral images; state-of-the-art results were achieved, e.g., improving the overall accuracy by 4.94% for the Salinas data. By considering the existence of unknown classes in the open world, our method achieved more accurate hyperspectral image classification, especially under the few-shot context.