Facing the Void: Overcoming Missing Data in Multi-View Imagery

TL;DR

This paper introduces a deep learning and metric learning-based method for multi-view image classification that is robust to missing data, improving accuracy across diverse datasets.

Contribution

It presents a novel technique that enhances multi-view classification robustness to missing data, adaptable to various applications and domains.

Findings

Improves classification accuracy over state-of-the-art methods

Effective on diverse multi-view aerial-ground datasets

Robust to missing data in multi-view scenarios

Abstract

In some scenarios, a single input image may not be enough to allow the object classification. In those cases, it is crucial to explore the complementary information extracted from images presenting the same object from multiple perspectives (or views) in order to enhance the general scene understanding and, consequently, increase the performance. However, this task, commonly called multi-view image classification, has a major challenge: missing data. In this paper, we propose a novel technique for multi-view image classification robust to this problem. The proposed method, based on state-of-the-art deep learning-based approaches and metric learning, can be easily adapted and exploited in other applications and domains. A systematic evaluation of the proposed algorithm was conducted using two multi-view aerial-ground datasets with very distinct properties. Results show that the proposed algorithm provides improvements in multi-view image classification accuracy when compared to state-of-the-art methods. Code available at \url{https://github.com/Gabriellm2003/remote_sensing_missing_data}.