Adaptive Online Mirror Descent for Tchebycheff Scalarization in Multi-Objective Learning

TL;DR

This paper introduces an adaptive online mirror descent algorithm for Tchebycheff scalarization in multi-objective learning, improving convergence and solution quality by addressing training oscillation and stagnation issues.

Contribution

It proposes (Ada)OMD-TCH, an adaptive algorithm with an online-to-batch conversion, offering better convergence rates and practical performance in multi-objective learning tasks.

Findings

Achieves a convergence rate of O(√(log m / T))

Effectively smooths training in synthetic and federated tasks

Produces diverse, fair, and preference-guided solutions

Abstract

Multi-objective learning (MOL) aims to learn under multiple potentially conflicting objectives and strike a proper balance. While recent preference-guided MOL methods often rely on additional optimization objectives or constraints, we consider the classic Tchebycheff scalarization (TCH) that naturally allows for locating solutions with user-specified trade-offs. Due to its minimax formulation, directly optimizing TCH often leads to training oscillation and stagnation. In light of this limitation, we propose an adaptive online mirror descent algorithm for TCH, called (Ada)OMD-TCH. One of our main ingredients is an adaptive online-to-batch conversion that significantly improves solution optimality over traditional conversion in practice while maintaining the same theoretical convergence guarantees. We show that (Ada)OMD-TCH achieves a convergence rate of $\mathcal O(\sqrt{\log m/T})$, where $m$ is the number of objectives and $T$ is the number of rounds, providing a tighter dependency on $m$ in the offline setting compared to existing work. Empirically, we demonstrate on both synthetic problems and federated learning tasks that (Ada)OMD-TCH effectively smooths the training process and yields preference-guided, specific, diverse, and fair solutions.