DeltaSeg: Tiered Attention and Deep Delta Learning for Multi-Class Structural Defect Segmentation

TL;DR

DeltaSeg introduces a novel multi-scale, attention-based segmentation architecture that effectively handles class imbalance and boundary precision in structural defect imagery, outperforming existing models.

Contribution

The paper proposes DeltaSeg, a new U-shaped network with tiered attention and deep delta learning, enhancing defect segmentation accuracy and robustness.

Findings

DeltaSeg outperforms 12 competing architectures on two defect datasets.

The model demonstrates strong generalization across damage types and imaging conditions.

Deep delta attention improves skip connection refinement and segmentation quality.

Abstract

Automated segmentation of structural defects from visual inspection imagery remains challenging due to the diversity of damage types, extreme class imbalance, and the need for precise boundary delineation. This paper presents DeltaSeg, a U-shaped encoder-decoder architecture with a tiered attention strategy that integrates Squeeze-and-Excitation (SE) channel attention in the encoder, Coordinate Attention at the bottleneck and decoder, and a novel Deep Delta Attention (DDA) mechanism in the skip connections. The encoder uses depthwise separable convolutions with dilated stages to maintain spatial resolution while expanding the receptive field. Atrous Spatial Pyramid Pooling (ASPP) at the bottleneck captures multi-scale context. The DDA module refines skip connections through a dual-path scheme combining a learned delta operator for nuisance feature suppression with spatial attention gates conditioned on decoder signals. Deep supervision through multi-scale auxiliary heads further strengthens gradient flow and encourages semantically meaningful features at intermediate decoder stages. We evaluate DeltaSeg on two datasets: the S2DS dataset (7 classes) and the Culvert-Sewer Defect Dataset (CSDD, 9 classes). Across both benchmarks, DeltaSeg consistently outperforms 12 competing architectures including U-Net, SA-UNet, UNet3+, SegFormer, Swin-UNet, EGE-UNet, FPN, and Mobile-UNETR, demonstrating strong generalization across damage types, imaging conditions, and structural geometries.