Robust and Active Learning for Deep Neural Network Regression

TL;DR

This paper introduces a gradient-based approach to identify local error maximizers in deep neural network regression models, enabling targeted fine-tuning or retraining through active learning with an oracle providing real-valued supervision.

Contribution

It presents a novel method for discovering local error regions in DNN regression, enhancing active learning and model accuracy with an efficient gradient-based technique.

Findings

Effective identification of local error maximizers

Improved model performance through targeted retraining

Potential reduction in oracle queries during training

Abstract

We describe a gradient-based method to discover local error maximizers of a deep neural network (DNN) used for regression, assuming the availability of an "oracle" capable of providing real-valued supervision (a regression target) for samples. For example, the oracle could be a numerical solver which, operationally, is much slower than the DNN. Given a discovered set of local error maximizers, the DNN is either fine-tuned or retrained in the manner of active learning.