Can Multitask Learning Enhance Model Explainability?

TL;DR

This paper proposes a multitask learning approach that uses satellite data modalities as auxiliary prediction targets to enhance model interpretability without sacrificing performance.

Contribution

It introduces a novel method leveraging auxiliary tasks in multimodal satellite data to intrinsically explain model behavior, improving interpretability and robustness.

Findings

Model performance remains comparable or better with auxiliary tasks.

Prediction errors can be explained via auxiliary task behavior.

Approach effective across segmentation, classification, and regression tasks.

Abstract

Remote sensing provides satellite data in diverse types and formats. The usage of multimodal learning networks exploits this diversity to improve model performance, except that the complexity of such networks comes at the expense of their interpretability. In this study, we explore how modalities can be leveraged through multitask learning to intrinsically explain model behavior. In particular, instead of additional inputs, we use certain modalities as additional targets to be predicted along with the main task. The success of this approach relies on the rich information content of satellite data, which remains as input modalities. We show how this modeling context provides numerous benefits: (1) in case of data scarcity, the additional modalities do not need to be collected for model inference at deployment, (2) the model performance remains comparable to the multimodal baseline performance, and in some cases achieves better scores, (3) prediction errors in the main task can be explained via the model behavior in the auxiliary task(s). We demonstrate the efficiency of our approach on three datasets, including segmentation, classification, and regression tasks. Code available at git.opendfki.de/hiba.najjar/mtl_explainability/.