L-SA: Learning Under-Explored Targets in Multi-Target Reinforcement Learning

TL;DR

This paper introduces L-SA, a framework that enhances multi-target reinforcement learning by adaptively sampling and actively querying under-explored targets, improving efficiency and success rates in visual navigation tasks.

Contribution

The paper proposes a novel L-SA framework combining adaptive sampling and active querying to address the Under-explored Target Problem in multi-target reinforcement learning.

Findings

L-SA improves sample efficiency and success rates in multi-target tasks.

Adaptive sampling and active querying synergistically enhance exploration of under-explored targets.

Experimental results demonstrate the effectiveness of L-SA in visual navigation tasks.

Abstract

Tasks that involve interaction with various targets are called multi-target tasks. When applying general reinforcement learning approaches for such tasks, certain targets that are difficult to access or interact with may be neglected throughout the course of training - a predicament we call Under-explored Target Problem (UTP). To address this problem, we propose L-SA (Learning by adaptive Sampling and Active querying) framework that includes adaptive sampling and active querying. In the L-SA framework, adaptive sampling dynamically samples targets with the highest increase of success rates at a high proportion, resulting in curricular learning from easy to hard targets. Active querying prompts the agent to interact more frequently with under-explored targets that need more experience or exploration. Our experimental results on visual navigation tasks show that the L-SA framework improves sample efficiency as well as success rates on various multi-target tasks with UTP. Also, it is experimentally demonstrated that the cyclic relationship between adaptive sampling and active querying effectively improves the sample richness of under-explored targets and alleviates UTP.