SODA: Self-organizing data augmentation in deep neural networks -- Application to biomedical image segmentation tasks

TL;DR

This paper introduces SODA, a self-organizing data augmentation method that dynamically allocates augmentation budgets during training, improving efficiency and potentially reducing computational costs in biomedical image segmentation tasks.

Contribution

It proposes a novel online learning approach to adaptively allocate data augmentation types during training, enhancing efficiency without significant additional computational cost.

Findings

Improved segmentation accuracy with adaptive augmentation.

Reduced computational time compared to uniform augmentation strategies.

Demonstrated effectiveness on biomedical image datasets.

Abstract

In practice, data augmentation is assigned a predefined budget in terms of newly created samples per epoch. When using several types of data augmentation, the budget is usually uniformly distributed over the set of augmentations but one can wonder if this budget should not be allocated to each type in a more efficient way. This paper leverages online learning to allocate on the fly this budget as part of neural network training. This meta-algorithm can be run at almost no extra cost as it exploits gradient based signals to determine which type of data augmentation should be preferred. Experiments suggest that this strategy can save computation time and thus goes in the way of greener machine learning practices.