SwIPE: Efficient and Robust Medical Image Segmentation with Implicit Patch Embeddings

TL;DR

SwIPE introduces a novel implicit patch embedding approach for medical image segmentation, achieving high accuracy, efficiency, and robustness by predicting shapes at the patch level instead of points or entire images.

Contribution

This work presents SwIPE, a new method that combines implicit neural representations with patch-level shape prediction for improved medical image segmentation.

Findings

Outperforms recent implicit approaches and state-of-the-art discrete methods.

Uses over 10x fewer parameters than comparable models.

Demonstrates superior data efficiency and robustness across datasets.

Abstract

Modern medical image segmentation methods primarily use discrete representations in the form of rasterized masks to learn features and generate predictions. Although effective, this paradigm is spatially inflexible, scales poorly to higher-resolution images, and lacks direct understanding of object shapes. To address these limitations, some recent works utilized implicit neural representations (INRs) to learn continuous representations for segmentation. However, these methods often directly adopted components designed for 3D shape reconstruction. More importantly, these formulations were also constrained to either point-based or global contexts, lacking contextual understanding or local fine-grained details, respectively--both critical for accurate segmentation. To remedy this, we propose a novel approach, SwIPE (Segmentation with Implicit Patch Embeddings), that leverages the advantages of INRs and predicts shapes at the patch level--rather than at the point level or image level--to enable both accurate local boundary delineation and global shape coherence. Extensive evaluations on two tasks (2D polyp segmentation and 3D abdominal organ segmentation) show that SwIPE significantly improves over recent implicit approaches and outperforms state-of-the-art discrete methods with over 10x fewer parameters. Our method also demonstrates superior data efficiency and improved robustness to data shifts across image resolutions and datasets. Code is available on Github (https://github.com/charzharr/miccai23-swipe-implicit-segmentation).