# Deep Bayesian Self-Training

## TL;DR

This paper introduces a Deep Bayesian Self-Training method that uses uncertainty estimates for automatic data annotation and adapts to dataset variability, improving performance over standard methods especially in safety-critical applications.

## Contribution

It presents a novel Bayesian self-training approach with uncertainty estimation and an adaptation procedure for diverse datasets, advancing automated data annotation techniques.

## Key findings

- Outperforms standard self-training baselines on multiple datasets.
- Uncertainty estimates improve safety-critical domain performance.
- Effective handling of high label variability between datasets.

## Abstract

Supervised Deep Learning has been highly successful in recent years, achieving state-of-the-art results in most tasks. However, with the ongoing uptake of such methods in industrial applications, the requirement for large amounts of annotated data is often a challenge. In most real world problems, manual annotation is practically intractable due to time/labour constraints, thus the development of automated and adaptive data annotation systems is highly sought after. In this paper, we propose both a (i) Deep Bayesian Self-Training methodology for automatic data annotation, by leveraging predictive uncertainty estimates using variational inference and modern Neural Network architectures, as well as (ii) a practical adaptation procedure for handling high label variability between different dataset distributions through clustering of Neural Network latent variable representations. An experimental study on both public and private datasets is presented illustrating the superior performance of the proposed approach over standard Self-Training baselines, highlighting the importance of predictive uncertainty estimates in safety-critical domains.