Addressing the Pitfalls of Image-Based Structural Health Monitoring: A Focus on False Positives, False Negatives, and Base Rate Bias

TL;DR

This paper critically examines the limitations of image-based structural health monitoring, especially false positives, false negatives, and base rate bias, and discusses strategies to improve reliability in real-world damage detection.

Contribution

It provides a comprehensive analysis of the impact of base rate bias on damage detection accuracy and proposes mitigation strategies for more reliable image-based SHM systems.

Findings

Bayesian and frequentist analyses reveal misleading results in low damage probability scenarios.

High-accuracy models can still produce false positives and negatives due to base rate bias.

Hybrid systems and improved training data can mitigate detection limitations.

Abstract

This study explores the limitations of image-based structural health monitoring (SHM) techniques in detecting structural damage. Leveraging machine learning and computer vision, image-based SHM offers a scalable and efficient alternative to manual inspections. However, its reliability is impacted by challenges such as false positives, false negatives, and environmental variability, particularly in low base rate damage scenarios. The Base Rate Bias plays a significant role, as low probabilities of actual damage often lead to misinterpretation of positive results. This study uses both Bayesian analysis and a frequentist approach to evaluate the precision of damage detection systems, revealing that even highly accurate models can yield misleading results when the occurrence of damage is rare. Strategies for mitigating these limitations are discussed, including hybrid systems that combine multiple data sources, human-in-the-loop approaches for critical assessments, and improving the quality of training data. These findings provide essential insights into the practical applicability of image-based SHM techniques, highlighting both their potential and their limitations for real-world infrastructure monitoring.