Towards General Purpose Robots at Scale: Lifelong Learning and Learning to Use Memory

TL;DR

This paper addresses the challenges of enabling robots to learn continuously and utilize memory effectively over long periods, proposing novel methods to improve lifelong learning and memory use in unstructured environments.

Contribution

It introduces t-DGR, a trajectory-based deep generative replay method, and a demonstration-based framework for memory utilization, advancing scalable lifelong learning in robotics.

Findings

t-DGR achieves state-of-the-art performance on Continual World benchmarks.

Demonstration-based memory teaching improves learning efficiency.

Framework enhances robot adaptability in unstructured environments.

Abstract

The widespread success of artificial intelligence in fields like natural language processing and computer vision has not yet fully transferred to robotics, where progress is hindered by the lack of large-scale training data and the complexity of real-world tasks. To address this, many robot learning researchers are pushing to get robots deployed at scale in everyday unstructured environments like our homes to initiate a data flywheel. While current robot learning systems are effective for certain short-horizon tasks, they are not designed to autonomously operate over long time horizons in unstructured environments. This thesis focuses on addressing two key challenges for robots operating over long time horizons: memory and lifelong learning. We propose two novel methods to advance these capabilities. First, we introduce t-DGR, a trajectory-based deep generative replay method that achieves state-of-the-art performance on Continual World benchmarks, advancing lifelong learning. Second, we develop a framework that leverages human demonstrations to teach agents effective memory utilization, improving learning efficiency and success rates on Memory Gym tasks. Finally, we discuss future directions for achieving the lifelong learning and memory capabilities necessary for robots to function at scale in real-world settings.