RoboBrain: Large-Scale Knowledge Engine for Robots
Ashutosh Saxena, Ashesh Jain, Ozan Sener, Aditya Jami, Dipendra K., Misra, Hema S. Koppula

TL;DR
RoboBrain is a large-scale knowledge engine that integrates diverse data modalities and sources to enhance robotic perception, language grounding, and planning capabilities, supporting complex task execution.
Contribution
This paper introduces RoboBrain, a novel knowledge engine that combines multiple data types and sources to improve robotic understanding and task performance.
Findings
Supports natural language grounding in robots
Enhances perception and planning tasks
Demonstrates integration of diverse knowledge sources
Abstract
In this paper we introduce a knowledge engine, which learns and shares knowledge representations, for robots to carry out a variety of tasks. Building such an engine brings with it the challenge of dealing with multiple data modalities including symbols, natural language, haptic senses, robot trajectories, visual features and many others. The \textit{knowledge} stored in the engine comes from multiple sources including physical interactions that robots have while performing tasks (perception, planning and control), knowledge bases from the Internet and learned representations from several robotics research groups. We discuss various technical aspects and associated challenges such as modeling the correctness of knowledge, inferring latent information and formulating different robotic tasks as queries to the knowledge engine. We describe the system architecture and how it supports…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Robot Manipulation and Learning · AI-based Problem Solving and Planning
