Towards Interpretable Multi-Task Learning Using Bilevel Programming

TL;DR

This paper introduces a bilevel programming approach for multi-task learning that learns sparse, interpretable task relationship graphs, enhancing model transparency without compromising predictive accuracy.

Contribution

It proposes a novel bilevel formulation for inducing sparse task relationship graphs in multi-task learning, with an efficient computation method and empirical validation.

Findings

Sparse graphs improve interpretability of task relationships.

The method maintains generalization performance.

Effective on both synthetic and real data.

Abstract

Interpretable Multi-Task Learning can be expressed as learning a sparse graph of the task relationship based on the prediction performance of the learned models. Since many natural phenomenon exhibit sparse structures, enforcing sparsity on learned models reveals the underlying task relationship. Moreover, different sparsification degrees from a fully connected graph uncover various types of structures, like cliques, trees, lines, clusters or fully disconnected graphs. In this paper, we propose a bilevel formulation of multi-task learning that induces sparse graphs, thus, revealing the underlying task relationships, and an efficient method for its computation. We show empirically how the induced sparse graph improves the interpretability of the learned models and their relationship on synthetic and real data, without sacrificing generalization performance. Code at https://bit.ly/GraphGuidedMTL