CoAction: Cross-task Correlation-aware Pareto Set Learning

TL;DR

CoAction introduces a transformer-based framework for multi-task Pareto set learning, efficiently capturing inter-task correlations and reducing computational costs in multi-objective optimization.

Contribution

It proposes a novel transformer-based approach that models inter-task correlations in Pareto set learning, enabling simultaneous multi-task optimization with improved efficiency.

Findings

Effective in capturing complex task dependencies.

Demonstrates competitive performance on benchmark and real-world problems.

Reduces computational costs by sharing knowledge across tasks.

Abstract

Pareto set learning (PSL) is an emerging paradigm in multi-objective optimization that trains neural networks to map preference vectors to Pareto optimal solutions. However, existing PSL methods primarily focus on solving a single multi-objective optimization problem at a time. This limitation not only increases computational costs in multi-objective multitask optimization scenarios by requiring a separate model for each task, but also fails to exploit the inter-task correlations across tasks. To address this, we propose a Cross-tAsk correlation-aware Pareto Set Learning (CoAction) framework, which leverages task-aware transformer to handle multiple tasks simultaneously. Specifically, by assigning task-specific embedding vectors to individual tasks, the model effectively distinguishes between tasks while facilitating knowledge sharing among them. We utilize a Transformer encoder as the backbone architecture to leverage its self-attention mechanism for capturing complex task dependencies. The proposed approach is evaluated on comprehensive multitask test suites covering both benchmark problems and real-world applications, demonstrating effectiveness and competitive performance in Hypervolume, Range, and Sparsity.