Regularising NARX models with multi-task learning

TL;DR

This paper introduces a multi-task learning approach to regularize NARX models, reducing overfitting and improving future prediction accuracy by jointly predicting current and future outputs.

Contribution

It proposes a novel multi-task learning framework for NARX models that enhances generalization and reduces overfitting in noisy environments.

Findings

MTL regularization lowers NMSE in high noise scenarios

Joint prediction of current and future outputs improves model robustness

Method outperforms independent NARX models in experiments

Abstract

A Nonlinear Auto-Regressive with eXogenous inputs (NARX) model can be used to describe time-varying processes; where the output depends on both previous outputs and current/previous external input variables. One limitation of NARX models is their propensity to overfit and result in poor generalisation for future predictions. The proposed method to help to overcome the issue of overfitting is a NARX model which predicts outputs at both the current time and several lead times into the future. This is a form of multi-task learner (MTL); whereby the lead time outputs will regularise the current time output. This work shows that for high noise level, MTL can be used to regularise NARX with a lower Normalised Mean Square Error (NMSE) compared to the NMSE of the independent learner counterpart.