Reducing the Long Tail Losses in Scientific Emulations with Active Learning

TL;DR

This paper introduces an active learning method using core-set selection and a warm start trick to reduce long tail losses in scientific emulation models, improving accuracy and efficiency.

Contribution

It presents a novel active learning approach with a warm start technique to effectively reduce long tail errors in scientific emulation tasks.

Findings

Achieved competitive performance with less labeled data.

Successfully reduced long tail losses in model training.

Demonstrated effectiveness across astrophysics and plasma physics case studies.

Abstract

Deep-learning-based models are increasingly used to emulate scientific simulations to accelerate scientific research. However, accurate, supervised deep learning models require huge amount of labelled data, and that often becomes the bottleneck in employing neural networks. In this work, we leveraged an active learning approach called core-set selection to actively select data, per a pre-defined budget, to be labelled for training. To further improve the model performance and reduce the training costs, we also warm started the training using a shrink-and-perturb trick. We tested on two case studies in different fields, namely galaxy halo occupation distribution modelling in astrophysics and x-ray emission spectroscopy in plasma physics, and the results are promising: we achieved competitive overall performance compared to using a random sampling baseline, and more importantly, successfully reduced the larger absolute losses, i.e. the long tail in the loss distribution, at virtually no overhead costs.