The Power of Certainty: How Confident Models Lead to Better Segmentation

TL;DR

This paper introduces a confidence-based self-distillation method for polyp segmentation that improves model performance and generalization without additional computational costs during testing.

Contribution

The proposed method enhances segmentation accuracy and generalization by using a confidence-driven self-distillation approach that requires only previous iteration data during training.

Findings

Outperforms state-of-the-art models in polyp segmentation

Generalizes well across multiple clinical datasets

Requires no extra computation during testing

Abstract

Deep learning models have been proposed for automatic polyp detection and precise segmentation of polyps during colonoscopy procedures. Although these state-of-the-art models achieve high performance, they often require a large number of parameters. Their complexity can make them prone to overfitting, particularly when trained on biased datasets, and can result in poor generalization across diverse datasets. Knowledge distillation and self-distillation are proposed as promising strategies to mitigate the limitations of large, over-parameterized models. These approaches, however, are resource-intensive, often requiring multiple models and significant memory during training. We propose a confidence-based self-distillation approach that outperforms state-of-the-art models by utilizing only previous iteration data storage during training, without requiring extra computation or memory usage during testing. Our approach calculates the loss between the previous and current iterations within a batch using a dynamic confidence coefficient. To evaluate the effectiveness of our approach, we conduct comprehensive experiments on the task of polyp segmentation. Our approach outperforms state-of-the-art models and generalizes well across datasets collected from multiple clinical centers. The code will be released to the public once the paper is accepted.