Toward Ethical AI Through Bayesian Uncertainty in Neural Question Answering

TL;DR

This paper investigates Bayesian methods to quantify uncertainty in neural question answering models, aiming to improve interpretability and ethical deployment by enabling models to abstain when uncertain.

Contribution

It introduces Bayesian inference techniques, including Laplace approximations, to neural question answering models, emphasizing uncertainty calibration over accuracy.

Findings

Bayesian methods improve uncertainty estimation in neural QA models.

Models can effectively abstain on uncertain predictions, enhancing interpretability.

Bayesian approaches contribute to more responsible AI deployment.

Abstract

We explore Bayesian reasoning as a means to quantify uncertainty in neural networks for question answering. Starting with a multilayer perceptron on the Iris dataset, we show how posterior inference conveys confidence in predictions. We then extend this to language models, applying Bayesian inference first to a frozen head and finally to LoRA-adapted transformers, evaluated on the CommonsenseQA benchmark. Rather than aiming for state-of-the-art accuracy, we compare Laplace approximations against maximum a posteriori (MAP) estimates to highlight uncertainty calibration and selective prediction. This allows models to abstain when confidence is low. An ``I don't know'' response not only improves interpretability but also illustrates how Bayesian methods can contribute to more responsible and ethical deployment of neural question-answering systems.