WildLMa: Long Horizon Loco-Manipulation in the Wild
Ri-Zhao Qiu, Yuchen Song, Xuanbin Peng, Sai Aneesh Suryadevara, Ge, Yang, Minghuan Liu, Mazeyu Ji, Chengzhe Jia, Ruihan Yang, Xueyan Zou,, Xiaolong Wang

TL;DR
WildLMa introduces a comprehensive framework combining learned skills, language-conditioned imitation, and planning for long-horizon, versatile mobile manipulation in diverse real-world environments using quadruped robots.
Contribution
The paper presents WildLMa, a novel system integrating a low-level controller, generalizable visuomotor skills, and LLM-based planning for in-the-wild manipulation tasks.
Findings
Higher grasping success rate with fewer demonstrations compared to RL baselines.
CLIP-based language-conditioned imitation generalizes to unseen objects.
Successful real-world applications like trash cleanup and object rearrangement.
Abstract
'In-the-wild' mobile manipulation aims to deploy robots in diverse real-world environments, which requires the robot to (1) have skills that generalize across object configurations; (2) be capable of long-horizon task execution in diverse environments; and (3) perform complex manipulation beyond pick-and-place. Quadruped robots with manipulators hold promise for extending the workspace and enabling robust locomotion, but existing results do not investigate such a capability. This paper proposes WildLMa with three components to address these issues: (1) adaptation of learned low-level controller for VR-enabled whole-body teleoperation and traversability; (2) WildLMa-Skill -- a library of generalizable visuomotor skills acquired via imitation learning or heuristics and (3) WildLMa-Planner -- an interface of learned skills that allow LLM planners to coordinate skills for long-horizon…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobotics and Sensor-Based Localization · Robotic Path Planning Algorithms · Modular Robots and Swarm Intelligence
MethodsLib · Contrastive Language-Image Pre-training
