Emergence of Different Modes of Tool Use in a Reaching and Dragging Task
Khuong Nguyen, Yoonsuck Choe

TL;DR
This study demonstrates that a deep reinforcement learning controller can spontaneously develop diverse and unexpected tool use behaviors, such as error correction and throwing, in a physics-based reaching and dragging task.
Contribution
It reveals the emergence of varied tool use strategies in a simulated environment without explicit programming, highlighting the richness of behaviors learned through reinforcement learning.
Findings
Emergence of diverse tool use behaviors like error correction and throwing.
Behavioral differences based on tool type and initial object position.
Rich repertoire of behaviors beyond basic motor primitives.
Abstract
Tool use is an important milestone in the evolution of intelligence. In this paper, we investigate different modes of tool use that emerge in a reaching and dragging task. In this task, a jointed arm with a gripper must grab a tool (T, I, or L-shaped) and drag an object down to the target location (the bottom of the arena). The simulated environment had real physics such as gravity and friction. We trained a deep-reinforcement learning based controller (with raw visual and proprioceptive input) with minimal reward shaping information to tackle this task. We observed the emergence of a wide range of unexpected behaviors, not directly encoded in the motor primitives or reward functions. Examples include hitting the object to the target location, correcting error of initial contact, throwing the tool toward the object, as well as normal expected behavior such as wide sweep. Also, we…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Robot Manipulation and Learning · Neural dynamics and brain function
