Task-Agnostic Learning to Accomplish New Tasks

TL;DR

This paper introduces TAL, a task-agnostic learning framework that leverages fragmented knowledge from environment interactions to adapt to new tasks, outperforming traditional RL and IL methods.

Contribution

The paper proposes a novel four-stage task-agnostic learning approach that uses a knowledge graph and action feature extractor to generalize to new tasks.

Findings

TAL outperforms state-of-the-art offline RL and IL methods by over 20%.

The method effectively learns fragmented knowledge from environment interactions.

Experiments demonstrate TAL's ability to adapt to new tasks in virtual indoor scenes.

Abstract

Reinforcement Learning (RL) and Imitation Learning (IL) have made great progress in robotic decision-making in recent years. However, these methods show obvious deterioration for new tasks that need to be completed through new combinations of actions. RL methods suffer from reward functions and distribution shifts, while IL methods are limited by expert demonstrations which do not cover new tasks. In contrast, humans can easily complete these tasks with the fragmented knowledge learned from task-agnostic experience. Inspired by this observation, this paper proposes a task-agnostic learning method (TAL for short) that can learn fragmented knowledge only from task-agnostic data to accomplish new tasks. TAL consists of four stages. First, the task-agnostic exploration is performed to collect data from interactions with the environment. The collected data is organized via a knowledge graph. Second, an action feature extractor is proposed and trained using the collected knowledge graph data for task-agnostic fragmented knowledge learning. Third, a candidate action generator is designed, which applies the action feature extractor on a new task to generate multiple candidate action sets. Finally, an action proposal network is designed to produce the probabilities for actions in a new task according to the environmental information. The probabilities are then used to generate order information for selecting actions to be executed from multiple candidate action sets to form the plan. Experiments on a virtual indoor scene show that the proposed method outperforms the state-of-the-art offline RL methods and IL methods by more than 20%.