Bayesian Uncertainty and Expected Gradient Length -- Regression: Two Sides Of The Same Coin?

TL;DR

This paper derives the formulation of Expected Gradient Length for regression, shows its equivalence to Bayesian uncertainty, and introduces EGL++ for human pose estimation, demonstrating its effectiveness on multiple datasets.

Contribution

The paper provides a theoretical derivation of Expected Gradient Length for regression, establishing its equivalence to Bayesian uncertainty, and proposes EGL++ for human pose estimation.

Findings

EGL++ is competitive with existing active learning methods for human pose estimation.

Expected Gradient Length in regression is theoretically equivalent to Bayesian uncertainty.

EGL++ effectively extends active learning to human pose estimation tasks.

Abstract

Active learning algorithms select a subset of data for annotation to maximize the model performance on a budget. One such algorithm is Expected Gradient Length, which as the name suggests uses the approximate gradient induced per example in the sampling process. While Expected Gradient Length has been successfully used for classification and regression, the formulation for regression remains intuitively driven. Hence, our theoretical contribution involves deriving this formulation, thereby supporting the experimental evidence. Subsequently, we show that expected gradient length in regression is equivalent to Bayesian uncertainty. If certain assumptions are infeasible, our algorithmic contribution (EGL++) approximates the effect of ensembles with a single deterministic network. Instead of computing multiple possible inferences per input, we leverage previously annotated samples to quantify the probability of previous labels being the true label. Such an approach allows us to extend expected gradient length to a new task: human pose estimation. We perform experimental validation on two human pose datasets (MPII and LSP/LSPET), highlighting the interpretability and competitiveness of EGL++ with different active learning algorithms for human pose estimation.