One Hyper-Initializer for All Network Architectures in Medical Image Analysis

TL;DR

This paper introduces a universal hyper-initializer that can pre-train and initialize any neural network architecture for medical image analysis, improving efficiency and reusability across various models and tasks.

Contribution

The proposed hyper-initializer is architecture-irrelevant and can be pre-trained once to initialize diverse network architectures in medical imaging tasks.

Findings

Effective across multiple medical imaging modalities

Reusable for different architectures and tasks

Enhances performance in data-limited scenarios

Abstract

Pre-training is essential to deep learning model performance, especially in medical image analysis tasks where limited training data are available. However, existing pre-training methods are inflexible as the pre-trained weights of one model cannot be reused by other network architectures. In this paper, we propose an architecture-irrelevant hyper-initializer, which can initialize any given network architecture well after being pre-trained for only once. The proposed initializer is a hypernetwork which takes a downstream architecture as input graphs and outputs the initialization parameters of the respective architecture. We show the effectiveness and efficiency of the hyper-initializer through extensive experimental results on multiple medical imaging modalities, especially in data-limited fields. Moreover, we prove that the proposed algorithm can be reused as a favorable plug-and-play initializer for any downstream architecture and task (both classification and segmentation) of the same modality.