Task Adaptation of Reinforcement Learning-based NAS Agents through Transfer Learning

TL;DR

This paper evaluates how transfer learning can improve the efficiency and effectiveness of reinforcement learning-based neural architecture search agents across different tasks, demonstrating significant benefits in training speed and performance.

Contribution

It introduces an assessment of transfer learning for RL-based NAS agents, showing pretraining enhances performance and reduces training time across tasks.

Findings

Pretraining benefits transfer performance in most tasks.

Transfer learning significantly shortens training time.

Effects vary depending on specific source and target tasks.

Abstract

Recently, a novel paradigm has been proposed for reinforcement learning-based NAS agents, that revolves around the incremental improvement of a given architecture. We assess the abilities of such reinforcement learning agents to transfer between different tasks. We perform our evaluation using the Trans-NASBench-101 benchmark, and consider the efficacy of the transferred agents, as well as how quickly they can be trained. We find that pretraining an agent on one task benefits the performance of the agent in another task in all but 1 task when considering final performance. We also show that the training procedure for an agent can be shortened significantly by pretraining it on another task. Our results indicate that these effects occur regardless of the source or target task, although they are more pronounced for some tasks than for others. Our results show that transfer learning can be an effective tool in mitigating the computational cost of the initial training procedure for reinforcement learning-based NAS agents.