# Sparse keypoint segmentation of lung fissures: efficient geometric deep learning for abstracting volumetric images

## TL;DR

This paper introduces a more efficient way to segment lung fissures in CT scans using geometric deep learning on sparse point clouds, offering faster processing with minimal loss in accuracy.

## Contribution

A novel pipeline using sparse keypoint clouds and a point cloud to mesh autoencoder (PC-AE) for efficient lung fissure segmentation is introduced.

## Key findings

- Graph convolutional networks (GCNs) provide a 21× speedup with only 1.4× increased error compared to 3D-CNNs.
- The proposed PC-AE is 3× faster than Poisson surface reconstruction with 1.5× increased error.
- The pipeline is more efficient for large-scale analyses and is available as open-source code.

## Abstract

Lung fissure segmentation on CT images often relies on 3D convolutional neural networks (CNNs). However, 3D-CNNs are inefficient for detecting thin structures like the fissures, which make up a tiny fraction of the entire image volume. We propose to make lung fissure segmentation more efficient by using geometric deep learning (GDL) on sparse point clouds.

We abstract image data with sparse keypoint (KP) clouds. We train GDL models to segment the point cloud, comparing three major paradigms of models (PointNets, graph convolutional networks (GCNs), and PointTransformers). From the sparse point segmentations, 3D meshes of the objects are reconstructed to obtain a dense surface. The state-of-the-art Poisson surface reconstruction (PSR) makes up most of the time in our pipeline. Therefore, we propose an efficient point cloud to mesh autoencoder (PC-AE) that deforms a template mesh to fit a point cloud in a single forward pass. Our pipeline is evaluated extensively and compared to the 3D-CNN gold standard nnU-Net on diverse clinical and pathological data.

GCNs yield the best trade-off between inference time and accuracy, being \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$21\times $$\end{document}21× faster with only \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$1.4\times $$\end{document}1.4× increased error over the nnU-Net. Our PC-AE also achieves a favorable trade-off, being \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$3\times $$\end{document}3× faster at \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$1.5\times $$\end{document}1.5× the error compared to the PSR.

We present a KP-based fissure segmentation pipeline that is more efficient than 3D-CNNs and can greatly speed up large-scale analyses. A novel PC-AE for efficient mesh reconstruction from sparse point clouds is introduced, showing promise not only for fissure segmentation. Source code is available on https://github.com/kaftanski/fissure-segmentation-IJCARS

The online version contains supplementary material available at 10.1007/s11548-024-03310-z.