Sharper Risk Bound for Multi-Task Learning with Multi-Graph Dependent Data

TL;DR

This paper introduces a new concentration inequality and analytical framework to derive sharper risk bounds for multi-task learning with graph-dependent data, improving theoretical guarantees from O(1/√n) to O(log n/n).

Contribution

It develops a Bennett-type inequality and a Talagrand-type inequality tailored for multi-graph dependent data, enabling tighter risk bounds in multi-task learning.

Findings

Achieves a risk bound of O(log n / n) for multi-task learning.

Validates theoretical improvements through experiments on Macro-AUC optimization.

Provides a new analytical framework for generalization analysis in graph-dependent data.

Abstract

In multi-task learning (MTL) with each task involving graph-dependent data, existing generalization analyses yield a \emph{sub-optimal} risk bound of $O(\frac{1}{\sqrt{n}})$, where $n$ is the number of training samples of each task. However, to improve the risk bound is technically challenging, which is attributed to the lack of a foundational sharper concentration inequality for multi-graph dependent random variables. To fill up this gap, this paper proposes a new Bennett-type inequality, enabling the derivation of a sharper risk bound of $O(\frac{\log n}{n})$. Technically, building on the proposed Bennett-type inequality, we propose a new Talagrand-type inequality for the empirical process, and further develop a new analytical framework of the local fractional Rademacher complexity to enhance generalization analyses in MTL with multi-graph dependent data. Finally, we apply the theoretical advancements to applications such as Macro-AUC optimization, illustrating the superiority of our theoretical results over prior work, which is also verified by experimental results.