Leveraging Foundation Models for Enhancing Robot Perception and Action

TL;DR

This paper explores how foundation models can be systematically utilized to improve robot perception, localization, interaction, and manipulation in unstructured environments, advancing semantics-aware robotic intelligence.

Contribution

It introduces a cohesive framework that leverages foundation models to enhance various aspects of robotic perception and action in complex environments.

Findings

Improved localization accuracy in unstructured settings

Enhanced interaction capabilities through semantics-aware models

Better manipulation performance in diverse environments

Abstract

This thesis investigates how foundation models can be systematically leveraged to enhance robotic capabilities, enabling more effective localization, interaction, and manipulation in unstructured environments. The work is structured around four core lines of inquiry, each addressing a fundamental challenge in robotics while collectively contributing to a cohesive framework for semantics-aware robotic intelligence.