TexLiverNet: Leveraging Medical Knowledge and Spatial-Frequency Perception for Enhanced Liver Tumor Segmentation

TL;DR

TexLiverNet is a novel model that combines lesion-specific textual annotations with advanced spatial-frequency perception to improve liver tumor segmentation accuracy, especially for small and boundary lesions.

Contribution

The paper introduces TexLiverNet, a new model that integrates lesion-specific text data with visual features using cross-attention and enhances spatial-frequency perception for better segmentation.

Findings

Outperforms existing state-of-the-art methods on multiple datasets.

Effectively delineates lesion boundaries and small lesions.

Reduces computational costs with agent-based cross-attention.

Abstract

Integrating textual data with imaging in liver tumor segmentation is essential for enhancing diagnostic accuracy. However, current multi-modal medical datasets offer only general text annotations, lacking lesion-specific details critical for extracting nuanced features, especially for fine-grained segmentation of tumor boundaries and small lesions. To address these limitations, we developed datasets with lesion-specific text annotations for liver tumors and introduced the TexLiverNet model. TexLiverNet employs an agent-based cross-attention module that integrates text features efficiently with visual features, significantly reducing computational costs. Additionally, enhanced spatial and adaptive frequency domain perception is proposed to precisely delineate lesion boundaries, reduce background interference, and recover fine details in small lesions. Comprehensive evaluations on public and private datasets demonstrate that TexLiverNet achieves superior performance compared to current state-of-the-art methods.