# Anatomy guided modality fusion for cancer segmentation in PET CT volumes and images

## TL;DR

This paper introduces a new method for combining CT and PET scans to better identify cancer areas, improving diagnosis and treatment planning.

## Contribution

The novel intermediate fusion approach with learnable normalization and anatomical guidance improves cancer segmentation in PET CT.

## Key findings

- The proposed method achieves a dice score of 0.8184 for cancer segmentation.
- The model reduces computational load with fewer filters while maintaining high accuracy.
- The HD95 score of 2.31 demonstrates improved spatial precision in tumor delineation.

## Abstract

Segmentation in computed tomography (CT) provides detailed anatomical information, while positron emission tomography (PET) provide the metabolic activity of cancer. Existing segmentation models in CT and PET either rely on early fusion, which struggles to effectively capture independent features from each modality, or late fusion, which is computationally expensive and fails to leverage the complementary nature of the two modalities. This research addresses the gap by proposing an intermediate fusion approach that optimally balances the strengths of both modalities. Our method leverages anatomical features to guide the fusion process while preserving spatial representation quality. We achieve this through the separate encoding of anatomical and metabolic features followed by an attentive fusion decoder. Unlike traditional fixed normalization techniques, we introduce novel “zero layers” with learnable normalization. The proposed intermediate fusion reduces the number of filters, resulting in a lightweight model. Our approach demonstrates superior performance, achieving a dice score of 0.8184 and an \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$\hbox {HD}^{95}$$\end{document} score of 2.31. The implications of this study include more precise tumor delineation, leading to enhanced cancer diagnosis and more effective treatment planning.