Orthogonal Ensemble Networks for Biomedical Image Segmentation

TL;DR

This paper introduces Orthogonal Ensemble Networks (OEN), a novel framework that enforces model diversity through orthogonal constraints, leading to improved performance and calibration in biomedical image segmentation tasks.

Contribution

The paper proposes a new orthogonal regularization method for ensemble learning, enhancing diversity and calibration in deep models for biomedical image segmentation.

Findings

OEN improves model calibration and robustness.

OEN outperforms baseline models in brain lesion segmentation.

Orthogonal constraints increase ensemble diversity and accuracy.

Abstract

Despite the astonishing performance of deep-learning based approaches for visual tasks such as semantic segmentation, they are known to produce miscalibrated predictions, which could be harmful for critical decision-making processes. Ensemble learning has shown to not only boost the performance of individual models but also reduce their miscalibration by averaging independent predictions. In this scenario, model diversity has become a key factor, which facilitates individual models converging to different functional solutions. In this work, we introduce Orthogonal Ensemble Networks (OEN), a novel framework to explicitly enforce model diversity by means of orthogonal constraints. The proposed method is based on the hypothesis that inducing orthogonality among the constituents of the ensemble will increase the overall model diversity. We resort to a new pairwise orthogonality constraint which can be used to regularize a sequential ensemble training process, resulting on improved predictive performance and better calibrated model outputs. We benchmark the proposed framework in two challenging brain lesion segmentation tasks --brain tumor and white matter hyper-intensity segmentation in MR images. The experimental results show that our approach produces more robust and well-calibrated ensemble models and can deal with challenging tasks in the context of biomedical image segmentation.