On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression

TL;DR

This paper analyzes the properties of mean absolute error (MAE) as a loss function for deep neural network vector-to-vector regression, demonstrating its advantages over mean squared error (MSE) through theoretical bounds and speech enhancement experiments.

Contribution

It presents new theoretical bounds for MAE in DNN regression and shows its suitability over MSE, supported by speech enhancement results.

Findings

MAE has a Lipschitz continuity property useful for performance bounds

MAE can be interpreted as modeling Laplacian errors

Speech enhancement experiments favor MAE over MSE

Abstract

In this paper, we exploit the properties of mean absolute error (MAE) as a loss function for the deep neural network (DNN) based vector-to-vector regression. The goal of this work is two-fold: (i) presenting performance bounds of MAE, and (ii) demonstrating new properties of MAE that make it more appropriate than mean squared error (MSE) as a loss function for DNN based vector-to-vector regression. First, we show that a generalized upper-bound for DNN-based vector- to-vector regression can be ensured by leveraging the known Lipschitz continuity property of MAE. Next, we derive a new generalized upper bound in the presence of additive noise. Finally, in contrast to conventional MSE commonly adopted to approximate Gaussian errors for regression, we show that MAE can be interpreted as an error modeled by Laplacian distribution. Speech enhancement experiments are conducted to corroborate our proposed theorems and validate the performance advantages of MAE over MSE for DNN based regression.