DeepMTL2R: A Library for Deep Multi-task Learning to Rank

TL;DR

DeepMTL2R is an open-source framework that uses transformer-based self-attention to optimize multiple relevance criteria simultaneously in ranking systems, enabling effective multi-task learning and trade-off analysis.

Contribution

It introduces a unified, scalable deep learning framework with 21 algorithms for multi-task ranking, supporting multi-objective optimization and complex dependency modeling.

Findings

Demonstrates competitive performance on a public dataset

Supports Pareto-optimal multi-objective ranking models

Visualizes trade-offs among relevance objectives

Abstract

This paper presents DeepMTL2R, an open-source deep learning framework for Multi-task Learning to Rank (MTL2R), where multiple relevance criteria must be optimized simultaneously. DeepMTL2R integrates heterogeneous relevance signals into a unified, context-aware model by leveraging the self-attention mechanism of transformer architectures, enabling effective learning across diverse and potentially conflicting objectives. The framework includes 21 state-of-the-art multi-task learning algorithms and supports multi-objective optimization to identify Pareto-optimal ranking models. By capturing complex dependencies and long-range interactions among items and labels, DeepMTL2R provides a scalable and expressive solution for modern ranking systems and facilitates controlled comparisons across MTL strategies. We demonstrate its effectiveness on a publicly available dataset, report competitive performance, and visualize the resulting trade-offs among objectives. DeepMTL2R is available at \href{https://github.com/amazon-science/DeepMTL2R}{https://github.com/amazon-science/DeepMTL2R}.