Neural Task Programming: Learning to Generalize Across Hierarchical Tasks

TL;DR

Neural Task Programming (NTP) is a hierarchical learning framework that enables robots to generalize across complex, compositional tasks by recursively decomposing demonstrations into sub-tasks and executing them with neural programs.

Contribution

The paper introduces NTP, a novel hierarchical neural program framework that combines few-shot learning and program induction for improved task generalization in robotics.

Findings

NTP generalizes well to unseen tasks with different structures.

NTP handles tasks with increasing complexity and length.

Experimental validation on three robot manipulation tasks.

Abstract

In this work, we propose a novel robot learning framework called Neural Task Programming (NTP), which bridges the idea of few-shot learning from demonstration and neural program induction. NTP takes as input a task specification (e.g., video demonstration of a task) and recursively decomposes it into finer sub-task specifications. These specifications are fed to a hierarchical neural program, where bottom-level programs are callable subroutines that interact with the environment. We validate our method in three robot manipulation tasks. NTP achieves strong generalization across sequential tasks that exhibit hierarchal and compositional structures. The experimental results show that NTP learns to generalize well to- wards unseen tasks with increasing lengths, variable topologies, and changing objectives.