Learning Where, What and How to Transfer: A Multi-Role Reinforcement Learning Approach for Evolutionary Multitasking

TL;DR

This paper introduces a reinforcement learning-based system for systematic knowledge transfer in evolutionary multitasking, addressing key challenges by learning policies for task selection, knowledge transfer, and transfer mechanisms, achieving state-of-the-art results.

Contribution

It presents a multi-role RL framework that learns generalizable transfer policies for EMT, integrating attention-based task matching and dynamic transfer control.

Findings

Achieves state-of-the-art performance on EMT benchmarks

Provides insights into learned transfer strategies

Demonstrates effective generalization across tasks

Abstract

Evolutionary multitasking (EMT) algorithms typically require tailored designs for knowledge transfer, in order to assure convergence and optimality in multitask optimization. In this paper, we explore designing a systematic and generalizable knowledge transfer policy through Reinforcement Learning. We first identify three major challenges: determining the task to transfer (where), the knowledge to be transferred (what) and the mechanism for the transfer (how). To address these challenges, we formulate a multi-role RL system where three (groups of) policy networks act as specialized agents: a task routing agent incorporates an attention-based similarity recognition module to determine source-target transfer pairs via attention scores; a knowledge control agent determines the proportion of elite solutions to transfer; and a group of strategy adaptation agents control transfer strength by dynamically controlling hyper-parameters in the underlying EMT framework. Through pre-training all network modules end-to-end over an augmented multitask problem distribution, a generalizable meta-policy is obtained. Comprehensive validation experiments show state-of-the-art performance of our method against representative baselines. Further in-depth analysis not only reveals the rationale behind our proposal but also provide insightful interpretations on what the system have learned.