# Toward Robust Uncertainty Estimation with Random Activation Functions

## TL;DR

This paper introduces Random Activation Functions (RAFs) Ensemble, a novel method that enhances uncertainty estimation in neural networks by diversifying models with different random activation functions, leading to improved robustness in out-of-distribution scenarios.

## Contribution

The paper proposes RAFs Ensemble, a new ensemble approach that increases diversity and robustness in uncertainty quantification by assigning different random activation functions to each network.

## Key findings

- Outperforms state-of-the-art ensemble uncertainty methods
- Demonstrates robustness on synthetic and real-world regression datasets
- Improves uncertainty estimation in out-of-distribution cases

## Abstract

Deep neural networks are in the limelight of machine learning with their excellent performance in many data-driven applications. However, they can lead to inaccurate predictions when queried in out-of-distribution data points, which can have detrimental effects especially in sensitive domains, such as healthcare and transportation, where erroneous predictions can be very costly and/or dangerous. Subsequently, quantifying the uncertainty of the output of a neural network is often leveraged to evaluate the confidence of its predictions, and ensemble models have proved to be effective in measuring the uncertainty by utilizing the variance of predictions over a pool of models. In this paper, we propose a novel approach for uncertainty quantification via ensembles, called Random Activation Functions (RAFs) Ensemble, that aims at improving the ensemble diversity toward a more robust estimation, by accommodating each neural network with a different (random) activation function. Extensive empirical study demonstrates that RAFs Ensemble outperforms state-of-the-art ensemble uncertainty quantification methods on both synthetic and real-world datasets in a series of regression tasks.