Interpretability in Convolutional Neural Networks for Building Damage Classification in Satellite Imagery

TL;DR

This paper develops an interpretable CNN-based approach for assessing building damage from satellite imagery post-disasters, identifying optimal loss functions and data modalities, and visualizing model focus areas to aid humanitarian efforts.

Contribution

It introduces a novel interpretable deep learning methodology for damage classification, optimizing loss functions, and visualizing model decision regions in satellite imagery.

Findings

Ordinal-cross entropy loss is most effective for training.

Including disaster type improves damage prediction accuracy.

Grad-CAM visualizations reveal model focus areas.

Abstract

Natural disasters ravage the world's cities, valleys, and shores on a regular basis. Deploying precise and efficient computational mechanisms for assessing infrastructure damage is essential to channel resources and minimize the loss of life. Using a dataset that includes labeled pre- and post- disaster satellite imagery, we take a machine learning-based remote sensing approach and train multiple convolutional neural networks (CNNs) to assess building damage on a per-building basis. We present a novel methodology of interpretable deep learning that seeks to explicitly investigate the most useful modalities of information in the training data to create an accurate classification model. We also investigate which loss functions best optimize these models. Our findings include that ordinal-cross entropy loss is the most optimal criterion for optimization to use and that including the type of disaster that caused the damage in combination with pre- and post-disaster training data most accurately predicts the level of damage caused. Further, we make progress in the qualitative representation of which parts of the images that the model is using to predict damage levels, through gradient-weighted class activation mapping (Grad-CAM). Our research seeks to computationally contribute to aiding in this ongoing and growing humanitarian crisis, heightened by anthropogenic climate change.