Output-weighted and relative entropy loss functions for deep learning precursors of extreme events

TL;DR

This paper introduces new loss functions tailored for deep learning models to better predict rare and extreme events in dynamical systems, addressing the challenge of imbalanced datasets.

Contribution

It proposes a novel output-weighted loss and extends relative entropy loss functions to low-dimensional systems, improving extreme event prediction accuracy.

Findings

Significant improvement in predicting extreme events across tested dynamical systems.

Enhanced loss functions better highlight rare events during training.

Applicable to systems with low-dimensional outputs.

Abstract

Many scientific and engineering problems require accurate models of dynamical systems with rare and extreme events. Such problems present a challenging task for data-driven modelling, with many naive machine learning methods failing to predict or accurately quantify such events. One cause for this difficulty is that systems with extreme events, by definition, yield imbalanced datasets and that standard loss functions easily ignore rare events. That is, metrics for goodness of fit used to train models are not designed to ensure accuracy on rare events. This work seeks to improve the performance of regression models for extreme events by considering loss functions designed to highlight outliers. We propose a novel loss function, the adjusted output weighted loss, and extend the applicability of relative entropy based loss functions to systems with low dimensional output. The proposed functions are tested using several cases of dynamical systems exhibiting extreme events and shown to significantly improve accuracy in predictions of extreme events.