Revisiting Rubik's Cube: Self-supervised Learning with Volume-wise Transformation for 3D Medical Image Segmentation

TL;DR

This paper introduces a novel self-supervised learning framework called Rubik's cube++ for 3D medical image segmentation, leveraging volume-wise transformations to improve network pre-training and reduce reliance on annotated data.

Contribution

It proposes a new context restoration task with volume-wise transformations for pre-training 3D neural networks on medical images, enhancing segmentation performance.

Findings

Pre-training with Rubik's cube++ improves segmentation accuracy.

The method outperforms training from scratch on multiple datasets.

Significant performance gains without extra annotated data.

Abstract

Deep learning highly relies on the quantity of annotated data. However, the annotations for 3D volumetric medical data require experienced physicians to spend hours or even days for investigation. Self-supervised learning is a potential solution to get rid of the strong requirement of training data by deeply exploiting raw data information. In this paper, we propose a novel self-supervised learning framework for volumetric medical images. Specifically, we propose a context restoration task, i.e., Rubik's cube++, to pre-train 3D neural networks. Different from the existing context-restoration-based approaches, we adopt a volume-wise transformation for context permutation, which encourages network to better exploit the inherent 3D anatomical information of organs. Compared to the strategy of training from scratch, fine-tuning from the Rubik's cube++ pre-trained weight can achieve better performance in various tasks such as pancreas segmentation and brain tissue segmentation. The experimental results show that our self-supervised learning method can significantly improve the accuracy of 3D deep learning networks on volumetric medical datasets without the use of extra data.