Twin Deformable Point Convolutions for Point Cloud Semantic Segmentation in Remote Sensing Scenes

TL;DR

This paper introduces Twin Deformable Point Convolutions (TDConvs), novel operators designed to adaptively learn features in remote sensing point clouds by leveraging their inherent latitude, longitude, and altitude arrangements, leading to improved segmentation.

Contribution

The paper proposes the first deformable convolution operators tailored for remote sensing point clouds, modeling their unique geographic coordinate characteristics for enhanced segmentation accuracy.

Findings

Achieves state-of-the-art segmentation performance on benchmark datasets.

Effectively models geographic features using cylinder-wise and sphere-wise deformable convolutions.

Outperforms existing methods in remote sensing point cloud segmentation.

Abstract

Thanks to the application of deep learning technology in point cloud processing of the remote sensing field, point cloud segmentation has become a research hotspot in recent years, which can be applied to real-world 3D, smart cities, and other fields. Although existing solutions have made unprecedented progress, they ignore the inherent characteristics of point clouds in remote sensing fields that are strictly arranged according to latitude, longitude, and altitude, which brings great convenience to the segmentation of point clouds in remote sensing fields. To consider this property cleverly, we propose novel convolution operators, termed Twin Deformable point Convolutions (TDConvs), which aim to achieve adaptive feature learning by learning deformable sampling points in the latitude-longitude plane and altitude direction, respectively. First, to model the characteristics of the latitude-longitude plane, we propose a Cylinder-wise Deformable point Convolution (CyDConv) operator, which generates a two-dimensional cylinder map by constructing a cylinder-like grid in the latitude-longitude direction. Furthermore, to better integrate the features of the latitude-longitude plane and the spatial geometric features, we perform a multi-scale fusion of the extracted latitude-longitude features and spatial geometric features, and realize it through the aggregation of adjacent point features of different scales. In addition, a Sphere-wise Deformable point Convolution (SpDConv) operator is introduced to adaptively offset the sampling points in three-dimensional space by constructing a sphere grid structure, aiming at modeling the characteristics in the altitude direction. Experiments on existing popular benchmarks conclude that our TDConvs achieve the best segmentation performance, surpassing the existing state-of-the-art methods.