A simple, strong baseline for building damage detection on the xBD dataset

TL;DR

This paper develops a simplified yet effective baseline for building damage detection on the xBD dataset, highlighting challenges in generalization due to dataset biases and class imbalance.

Contribution

The authors create a simplified, strong baseline model for building damage detection that retains performance while being more applicable and easier to transfer across datasets.

Findings

Simplified model performs comparably to complex solutions.

Both models struggle to generalize to unseen locations.

Dataset biases and class imbalance affect model generalization.

Abstract

We construct a strong baseline method for building damage detection by starting with the highly-engineered winning solution of the xView2 competition, and gradually stripping away components. This way, we obtain a much simpler method, while retaining adequate performance. We expect the simplified solution to be more widely and easily applicable. This expectation is based on the reduced complexity, as well as the fact that we choose hyperparameters based on simple heuristics, that transfer to other datasets. We then re-arrange the xView2 dataset splits such that the test locations are not seen during training, contrary to the competition setup. In this setting, we find that both the complex and the simplified model fail to generalize to unseen locations. Analyzing the dataset indicates that this failure to generalize is not only a model-based problem, but that the difficulty might also be influenced by the unequal class distributions between events. Code, including the baseline model, is available under https://github.com/PaulBorneP/Xview2_Strong_Baseline