Complex Wavelet Mutual Information Loss: A Multi-Scale Loss Function for Semantic Segmentation

TL;DR

This paper introduces the complex wavelet mutual information (CWMI) loss, a multi-scale loss function leveraging complex steerable pyramid decomposition and mutual information to improve semantic segmentation accuracy and boundary detection.

Contribution

The paper proposes a novel CWMI loss that captures multi-scale and directional features efficiently, enhancing segmentation performance over existing pixel-wise and regional loss functions.

Findings

CWMI loss improves pixel accuracy and topological metrics.

It achieves these improvements with minimal computational overhead.

Experimental results outperform state-of-the-art methods.

Abstract

Recent advancements in deep neural networks have significantly enhanced the performance of semantic segmentation. However, class imbalance and instance imbalance remain persistent challenges, where smaller instances and thin boundaries are often overshadowed by larger structures. To address the multiscale nature of segmented objects, various models have incorporated mechanisms such as spatial attention and feature pyramid networks. Despite these advancements, most loss functions are still primarily pixel-wise, while regional and boundary-focused loss functions often incur high computational costs or are restricted to small-scale regions. To address this limitation, we propose the complex wavelet mutual information (CWMI) loss, a novel loss function that leverages mutual information from subband images decomposed by a complex steerable pyramid. The complex steerable pyramid captures features across multiple orientations and preserves structural similarity across scales. Meanwhile, mutual information is well-suited to capturing high-dimensional directional features and offers greater noise robustness. Extensive experiments on diverse segmentation datasets demonstrate that CWMI loss achieves significant improvements in both pixel-wise accuracy and topological metrics compared to state-of-the-art methods, while introducing minimal computational overhead. Our code is available at https://github.com/lurenhaothu/CWMI