Adaptive Detector-Verifier Framework for Zero-Shot Polyp Detection in Open-World Settings

TL;DR

This paper introduces AdaptiveDetector, a two-stage framework combining a YOLOv11 detector and a vision-language verifier, designed to improve zero-shot polyp detection in challenging real-world endoscopy conditions by adaptively adjusting confidence thresholds and using cost-sensitive reinforcement learning.

Contribution

The work presents a novel adaptive detector-verifier framework with a cost-sensitive verifier fine-tuned via reinforcement learning, addressing domain gaps in polyp detection under adverse conditions.

Findings

Improves recall by 14-22 percentage points over baseline YOLO.

Maintains precision within 0.7 to 1.7 points of baseline.

Demonstrates effectiveness in synthetic adverse conditions for zero-shot evaluation.

Abstract

Polyp detectors trained on clean datasets often underperform in real-world endoscopy, where illumination changes, motion blur, and occlusions degrade image quality. Existing approaches struggle with the domain gap between controlled laboratory conditions and clinical practice, where adverse imaging conditions are prevalent. In this work, we propose AdaptiveDetector, a novel two-stage detector-verifier framework comprising a YOLOv11 detector with a vision-language model (VLM) verifier. The detector adaptively adjusts per-frame confidence thresholds under VLM guidance, while the verifier is fine-tuned with Group Relative Policy Optimization (GRPO) using an asymmetric, cost-sensitive reward function specifically designed to discourage missed detections -- a critical clinical requirement. To enable realistic assessment under challenging conditions, we construct a comprehensive synthetic testbed by systematically degrading clean datasets with adverse conditions commonly encountered in clinical practice, providing a rigorous benchmark for zero-shot evaluation. Extensive zero-shot evaluation on synthetically degraded CVC-ClinicDB and Kvasir-SEG images demonstrates that our approach improves recall by 14 to 22 percentage points over YOLO alone, while precision remains within 0.7 points below to 1.7 points above the baseline. This combination of adaptive thresholding and cost-sensitive reinforcement learning achieves clinically aligned, open-world polyp detection with substantially fewer false negatives, thereby reducing the risk of missed precancerous polyps and improving patient outcomes.