Are Foundation Models the Route to Full-Stack Transfer in Robotics?

TL;DR

This paper reviews how foundation models and transformer networks influence transfer learning in robotics across various abstraction levels, emphasizing their potential to enable comprehensive transfer capabilities.

Contribution

It provides an overview of foundation models' impact on robotic transfer learning, highlighting common concepts and discussing challenges in data and benchmarking.

Findings

Foundation models advance transfer learning in robotics.

They facilitate high-level and low-level transfer capabilities.

Challenges remain in data collection and benchmarking for robotics.

Abstract

In humans and robots alike, transfer learning occurs at different levels of abstraction, from high-level linguistic transfer to low-level transfer of motor skills. In this article, we provide an overview of the impact that foundation models and transformer networks have had on these different levels, bringing robots closer than ever to "full-stack transfer". Considering LLMs, VLMs and VLAs from a robotic transfer learning perspective allows us to highlight recurring concepts for transfer, beyond specific implementations. We also consider the challenges of data collection and transfer benchmarks for robotics in the age of foundation models. Are foundation models the route to full-stack transfer in robotics? Our expectation is that they will certainly stay on this route as a key technology.