TP-Seg: Task-Prototype Framework for Unified Medical Lesion Segmentation

TL;DR

TP-Seg is a unified framework for medical lesion segmentation that uses task prototypes and adapters to improve performance across diverse tasks and modalities.

Contribution

It introduces a novel task-prototype framework with adaptive feature extraction and semantic anchors for improved medical lesion segmentation.

Findings

Outperforms existing methods on 8 medical lesion segmentation tasks.

Demonstrates strong generalization and scalability.

Effective across multiple imaging modalities.

Abstract

Building a unified model with a single set of parameters to efficiently handle diverse types of medical lesion segmentation has become a crucial objective for AI-assisted diagnosis. Existing unified segmentation approaches typically rely on shared encoders across heterogeneous tasks and modalities, which often leads to feature entanglement, gradient interference, and suboptimal lesion discrimination. In this work, we propose TP-Seg, a task-prototype framework for unified medical lesion segmentation. On one hand, the task-conditioned adapter effectively balances shared and task-specific representations through a dual-path expert structure, enabling adaptive feature extraction across diverse medical imaging modalities and lesion types. On the other hand, the prototype-guided task decoder introduces learnable task prototypes as semantic anchors and employs a cross-attention mechanism to achieve fine-grained modeling of task-specific foreground and background semantics. Without bells and whistles, TP-Seg consistently outperforms specialized, general and unified segmentation methods across 8 different medical lesion segmentation tasks covering multiple imaging modalities, demonstrating strong generalization, scalability and clinical applicability.