Towards Segmenting the Invisible: An End-to-End Registration and Segmentation Framework for Weakly Supervised Tumour Analysis

TL;DR

This paper explores the use of a hybrid registration-segmentation framework for weakly supervised tumour analysis across MRI and CT modalities, revealing limitations when pathology is invisible in the target modality.

Contribution

It introduces a novel end-to-end registration and segmentation framework combining MSCGUNet and UNet, and analyzes the challenges of cross-modality label transfer for invisible tumours.

Findings

Successful registration and segmentation of healthy liver anatomy with a Dice score of 0.72.

Significant performance drop to a Dice score of 0.16 on clinical tumour data.

Identification of fundamental limitations in registration-based label transfer for invisible pathology.

Abstract

Liver tumour ablation presents a significant clinical challenge: whilst tumours are clearly visible on pre-operative MRI, they are often effectively invisible on intra-operative CT due to minimal contrast between pathological and healthy tissue. This work investigates the feasibility of cross-modality weak supervision for scenarios where pathology is visible in one modality (MRI) but absent in another (CT). We present a hybrid registration-segmentation framework that combines MSCGUNet for inter-modal image registration with a UNet-based segmentation module, enabling registration-assisted pseudo-label generation for CT images. Our evaluation on the CHAOS dataset demonstrates that the pipeline can successfully register and segment healthy liver anatomy, achieving a Dice score of 0.72. However, when applied to clinical data containing tumours, performance degrades substantially (Dice score of 0.16), revealing the fundamental limitations of current registration methods when the target pathology lacks corresponding visual features in the target modality. We analyse the "domain gap" and "feature absence" problems, demonstrating that whilst spatial propagation of labels via registration is feasible for visible structures, segmenting truly invisible pathology remains an open challenge. Our findings highlight that registration-based label transfer cannot compensate for the absence of discriminative features in the target modality, providing important insights for future research in cross-modality medical image analysis. Code an weights are available at: https://github.com/BudhaTronix/Weakly-Supervised-Tumour-Detection