Towards Automated Post-Earthquake Inspections with Deep Learning-based Condition-Aware Models

TL;DR

This paper presents a deep learning framework that combines semantic segmentation and 3D modeling to automate and accelerate post-earthquake structural inspections using UAV imagery.

Contribution

It introduces a novel method to generate condition-aware 3D models from UAV images by projecting deep learning inferences onto 3D meshes for rapid damage assessment.

Findings

Successful implementation on a damaged building after the 2017 Mexico Earthquake

Qualitative evaluation shows promising results for automated inspections

Framework integrates deep learning with 3D modeling for damage detection

Abstract

In the aftermath of an earthquake, rapid structural inspections are required to get citizens back in to their homes and offices in a safe and timely manner. These inspections gfare typically conducted by municipal authorities through structural engineer volunteers. As manual inspec-tions can be time consuming, laborious and dangerous, research has been underway to develop methods to help speed up and increase the automation of the entire process. Researchers typi-cally envisage the use of unmanned aerial vehicles (UAV) for data acquisition and computer vision for data processing to extract actionable information. In this work we propose a new framework to generate vision-based condition-aware models that can serve as the basis for speeding up or automating higher level inspection decisions. The condition-aware models are generated by projecting the inference of trained deep-learning models on a set of images of a structure onto a 3D mesh model generated through multi-view stereo from the same image set. Deep fully convolutional residual networks are used for semantic segmentation of images of buildings to provide (i) damage information such as cracks and spalling (ii) contextual infor-mation such as the presence of a building and visually identifiable components like windows and doors. The proposed methodology was implemented on a damaged building that was sur-veyed by the authors after the Central Mexico Earthquake in September 2017 and qualitative-ly evaluated. Results demonstrate the promise of the proposed method towards the ultimate goal of rapid and automated post-earthquake inspections.