Two out of Three (ToT): using self-consistency to make robust predictions

TL;DR

This paper introduces the ToT algorithm, inspired by human conflict detection, which enhances deep learning robustness by generating multiple predictions and abstaining when uncertainty is high.

Contribution

The paper proposes the ToT method that improves DL robustness by leveraging self-consistency and abstention based on conflicting predictions.

Findings

ToT increases decision reliability in uncertain scenarios.

The method effectively identifies when models should abstain from uncertain predictions.

Experimental results demonstrate improved robustness over baseline models.

Abstract

Deep learning (DL) can automatically construct intelligent agents, deep neural networks (alternatively, DL models), that can outperform humans in certain tasks. However, the operating principles of DL remain poorly understood, making its decisions incomprehensible. As a result, it poses a great risk to deploy DL in high-stakes domains in which mistakes or errors may lead to critical consequences. Here, we aim to develop an algorithm that can help DL models make more robust decisions by allowing them to abstain from answering when they are uncertain. Our algorithm, named `Two out of Three (ToT)', is inspired by the sensitivity of the human brain to conflicting information. ToT creates two alternative predictions in addition to the original model prediction and uses the alternative predictions to decide whether it should provide an answer or not.