PreCM: The Padding-based Rotation Equivariant Convolution Mode for Semantic Segmentation

TL;DR

This paper introduces PreCM, a padding-based rotation equivariant convolution mode, enhancing CNNs' ability to handle arbitrarily rotated images in semantic segmentation, with improved accuracy and a new evaluation metric.

Contribution

The paper proposes a universal rotation equivariant convolution framework and a novel padding-based convolution mode applicable to various convolution types, improving rotation robustness in segmentation networks.

Findings

PreCM-based networks show significant IOU improvements under random rotations.

The Rotation Difference metric effectively quantifies rotation robustness.

PreCM reduces the impact of orientation variations in segmentation tasks.

Abstract

Semantic segmentation is an important branch of image processing and computer vision. With the popularity of deep learning, various convolutional neural networks have been proposed for pixel-level classification and segmentation tasks. In practical scenarios, however, imaging angles are often arbitrary, encompassing instances such as water body images from remote sensing and capillary and polyp images in the medical domain, where prior orientation information is typically unavailable to guide these networks to extract more effective features. In this case, learning features from objects with diverse orientation information poses a significant challenge, as the majority of CNN-based semantic segmentation networks lack rotation equivariance to resist the disturbance from orientation information. To address this challenge, this paper first constructs a universal convolution-group framework aimed at more fully utilizing orientation information and equipping the network with rotation equivariance. Subsequently, we mathematically design a padding-based rotation equivariant convolution mode (PreCM), which is not only applicable to multi-scale images and convolutional kernels but can also serve as a replacement component for various types of convolutions, such as dilated convolutions, transposed convolutions, and asymmetric convolution. To quantitatively assess the impact of image rotation in semantic segmentation tasks, we also propose a new evaluation metric, Rotation Difference (RD). The replacement experiments related to six existing semantic segmentation networks on three datasets show that, the average Intersection Over Union (IOU) of their PreCM-based versions respectively improve 6.91%, 10.63%, 4.53%, 5.93%, 7.48%, 8.33% compared to their original versions in terms of random angle rotation. And the average RD values are decreased by 3.58%, 4.56%, 3.47%, 3.66%, 3.47%, 3.43% respectively.