Divergent Ensemble Networks: Enhancing Uncertainty Estimation with Shared Representations and Independent Branching

TL;DR

Divergent Ensemble Networks (DEN) improve uncertainty estimation in neural networks by combining shared representations with independent branches, reducing redundancy and enhancing efficiency.

Contribution

The paper introduces DEN, a novel ensemble architecture that balances shared feature learning with independent branches for better efficiency and diversity.

Findings

Reduces parameter redundancy compared to traditional ensembles.

Maintains ensemble diversity through independent branching.

Enhances scalability and computational efficiency.

Abstract

Ensemble learning has proven effective in improving predictive performance and estimating uncertainty in neural networks. However, conventional ensemble methods often suffer from redundant parameter usage and computational inefficiencies due to entirely independent network training. To address these challenges, we propose the Divergent Ensemble Network (DEN), a novel architecture that combines shared representation learning with independent branching. DEN employs a shared input layer to capture common features across all branches, followed by divergent, independently trainable layers that form an ensemble. This shared-to-branching structure reduces parameter redundancy while maintaining ensemble diversity, enabling efficient and scalable learning.