Detecting Damage Building Using Real-time Crowdsourced Images and Transfer Learning

TL;DR

This paper develops a transfer learning-based deep learning model that automatically detects damaged buildings from social media images post-earthquake, enabling rapid damage assessment and visualization of decision factors.

Contribution

It introduces a novel approach using transfer learning and crowdsourced images to identify earthquake-damaged buildings in real-time from social media posts.

Findings

Achieved high accuracy in detecting damaged buildings.

Operates in near real-time on Twitter data.

Visualizes decision-making process with Grad-CAM.

Abstract

After significant earthquakes, we can see images posted on social media platforms by individuals and media agencies owing to the mass usage of smartphones these days. These images can be utilized to provide information about the shaking damage in the earthquake region both to the public and research community, and potentially to guide rescue work. This paper presents an automated way to extract the damaged building images after earthquakes from social media platforms such as Twitter and thus identify the particular user posts containing such images. Using transfer learning and ~6500 manually labelled images, we trained a deep learning model to recognize images with damaged buildings in the scene. The trained model achieved good performance when tested on newly acquired images of earthquakes at different locations and ran in near real-time on Twitter feed after the 2020 M7.0 earthquake in Turkey. Furthermore, to better understand how the model makes decisions, we also implemented the Grad-CAM method to visualize the important locations on the images that facilitate the decision.