Building Inspection Toolkit: Unified Evaluation and Strong Baselines for Damage Recognition

TL;DR

The paper introduces bikit, a comprehensive toolkit and leaderboard for damage recognition in building inspections, providing standardized datasets, evaluation metrics, and strong baselines to foster consistent research and comparison.

Contribution

It presents bikit, a unified data hub with standardized datasets, evaluation splits, metrics, and strong baseline models for damage recognition in building inspections.

Findings

Bikit offers a consistent benchmark for damage recognition tasks.

Strong baseline models are established using transfer learning approaches.

The toolkit facilitates fair comparison and accelerates research in the domain.

Abstract

In recent years, several companies and researchers have started to tackle the problem of damage recognition within the scope of automated inspection of built structures. While companies are neither willing to publish associated data nor models, researchers are facing the problem of data shortage on one hand and inconsistent dataset splitting with the absence of consistent metrics on the other hand. This leads to incomparable results. Therefore, we introduce the building inspection toolkit -- bikit -- which acts as a simple to use data hub containing relevant open-source datasets in the field of damage recognition. The datasets are enriched with evaluation splits and predefined metrics, suiting the specific task and their data distribution. For the sake of compatibility and to motivate researchers in this domain, we also provide a leaderboard and the possibility to share model weights with the community. As starting point we provide strong baselines for multi-target classification tasks utilizing extensive hyperparameter search using three transfer learning approaches for state-of-the-art algorithms. The toolkit and the leaderboard are available online.