NuNext: Reframing Nucleus Detection as Next-Point Detection

TL;DR

NuNext introduces a novel approach to nucleus detection in histopathology by reframing it as next-point prediction, leveraging a multimodal large language model with a two-stage training process to improve accuracy and robustness.

Contribution

The paper presents a new formulation of nucleus detection as next-point prediction and develops a multimodal large language model trained with innovative supervision and reinforcement strategies.

Findings

Outperforms existing methods on nine benchmarks.

Effective in detecting nucleus centroids with high accuracy.

Robust across diverse histopathology datasets.

Abstract

Nucleus detection in histopathology is pivotal for a wide range of clinical applications. Existing approaches either regress nuclear proxy maps that require complex post-processing, or employ dense anchors or queries that introduce severe foreground-background imbalance. In this work, we reformulate nucleus detection as next-point prediction, wherein a multimodal large language model is developed to directly output foreground nucleus centroids from the input image. The model is trained in two stages. In the supervised learning stage, we propose spatial-aware soft supervision to relax strict centroid matching and a chain-of-visual-thought strategy to incorporate visual priors that facilitate coordinate prediction. In the reinforcement fine-tuning stage, we design distribution matching reward, low-variance group filtering, and fine-grained advantage shaping to further improve the model's detection quality. Extensive experiments on nine widely used benchmarks demonstrate the superiority of our method. Code will be released soon.