A graph generation pipeline for critical infrastructures based on heuristics, images and depth data

TL;DR

This paper introduces a cost-effective graph generation pipeline for critical infrastructure models using photogrammetry, deep learning, and heuristics, enabling accurate and transparent representations from RGB and depth data.

Contribution

The novel pipeline combines deep learning and user-defined heuristics to generate infrastructure graphs from RGB images and depth data, reducing costs and increasing transparency.

Findings

Produced graphs closely match ground truth in hydraulic systems

Pipeline is adaptable to various infrastructure types

Enhances transparency for high-stakes decision-making

Abstract

Virtual representations of physical critical infrastructures, such as water or energy plants, are used for simulations and digital twins to ensure resilience and continuity of their services. These models usually require 3D point clouds from laser scanners that are expensive to acquire and require specialist knowledge to use. In this article, we present a prototypical graph generation pipeline based on photogrammetry. The pipeline detects relevant objects and predicts their relation using RGB images and depth data generated by a stereo camera. This more cost-effective approach uses deep learning for object detection and instance segmentation of the objects, and employs user-defined heuristics or rules to infer their relations. Results of two hydraulic systems show that this strategy can produce graphs close to the ground truth. While this study focuses on hydraulic systems, the general process can be used to tailor the method to other types of infrastructures and applications. The user-defined rules create transparency qualifying the pipeline to be used in the high stakes decision-making that is required for critical infrastructures.