# Deep learning for deep learning performance: How much data is needed for segmentation in biomedical imaging?

## TL;DR

This paper introduces a framework to estimate the minimal dataset size needed for stable performance in biomedical image segmentation using deep learning.

## Contribution

A DL-specific framework for estimating minimal dataset size and a surrogate LSTM model to predict segmentation performance are proposed.

## Key findings

- Segmentation performance improves with more data but plateaus around 80% data usage.
- A simple LSTM model accurately predicts final DSC with low error across 2D and 3D tasks.
- High-quality moderate datasets can yield clinically viable models without excessive data collection.

## Abstract

Deep learning (DL) models are widely adopted in biomedical imaging, where image segmentation is increasingly recognized as a quantitative tool for extracting clinically meaningful information. However, model performance critically depends on dataset size and training configuration, including model capacity. Traditional sample size estimation methods are inadequate for DL due to its reliance on high-dimensional data and its nonlinear learning behavior. To address this gap, we propose a DL-specific framework to estimate the minimal dataset size required for stable segmentation performance. We validate this framework across two distinct clinical tasks: colorectal polyp segmentation from 2D endoscopic images (Kvasir-SEG) and glioma segmentation from 3D brain MRIs (BraTS 2020). We trained residual U-Nets—a simple, yet foundational architecture—across 200 configurations for Kvasir-SEG and 40 configurations for BraTS 2020, varying data subsets (2%–100% for the 2D task and 5%–100% for the 3D task). In both tasks, performance metrics such as the Dice Similarity Coefficient (DSC) consistently improved with increasing data and depth, but gains invariably plateaued beyond approximately 80% data usage. The best configuration for polyp segmentation (6 layers, 100% data) achieved a DSC of 0.86, while the best for brain tumor segmentation reached a DSC of 0.79. Critically, we introduce a surrogate modeling pipeline using Long Short-Term Memory (LSTM) networks to predict these performance curves. A simple uni-directional LSTM model accurately forecasted the final DSC, accurately forecasting the final DSC with low mean absolute error across both tasks. These findings demonstrate that segmentation performance can be reliably estimated with lightweight models, suggesting that collecting a moderate amount of high-quality data is often sufficient for developing clinically viable DL models. Our framework provides a practical, empirical method for optimizing resource allocation in medical AI development.