BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment
Zeyi Liu, Zhenjia Xu, Shuran Song

TL;DR
BusyBot is a comprehensive learning framework that enables a robot to interact, reason about, and plan in complex environments with diverse objects, demonstrating generalization to unseen scenarios in both simulation and real-world settings.
Contribution
The paper introduces BusyBoard, a new environment for rich visual feedback, and BusyBot, a self-supervised framework for integrated interaction, reasoning, and planning in robotic manipulation.
Findings
BusyBot effectively learns interaction policies in complex environments.
It successfully reasons about inter-object relations through causal discovery.
The approach generalizes to unseen objects and relations in real-world tests.
Abstract
We introduce BusyBoard, a toy-inspired robot learning environment that leverages a diverse set of articulated objects and inter-object functional relations to provide rich visual feedback for robot interactions. Based on this environment, we introduce a learning framework, BusyBot, which allows an agent to jointly acquire three fundamental capabilities (interaction, reasoning, and planning) in an integrated and self-supervised manner. With the rich sensory feedback provided by BusyBoard, BusyBot first learns a policy to efficiently interact with the environment; then with data collected using the policy, BusyBot reasons the inter-object functional relations through a causal discovery network; and finally by combining the learned interaction policy and relation reasoning skill, the agent is able to perform goal-conditioned manipulation tasks. We evaluate BusyBot in both simulated and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Topic Modeling · Natural Language Processing Techniques
