Update-Free On-Policy Steering via Verifiers
Maria Attarian, Ian Vyse, Claas Voelcker, Jasper Gerigk, Evgenii Opryshko, Anas Almasri, Sumeet Singh, Yilun Du, Igor Gilitschenski

TL;DR
This paper introduces UF-OPS, a lightweight, update-free on-policy steering method that uses verifiers trained on rollout data to improve robot manipulation success rates without altering the base policy.
Contribution
The paper proposes UF-OPS, a novel method that enhances on-policy policies with verifiers trained during initial evaluation, enabling success likelihood prediction and strategy adaptation at execution time.
Findings
Achieved 49% average success rate improvement over base policy.
Validated effectiveness in both simulation and real-world tasks.
Method is lightweight and does not modify base policy parameters.
Abstract
In recent years, Behavior Cloning (BC) has become one of the most prevalent methods for enabling robots to mimic human demonstrations. However, despite their successes, BC policies are often brittle and struggle with precise manipulation. To overcome these issues, we propose UF-OPS, an Update-Free On-Policy Steering method that enables the robot to predict the success likelihood of its actions and adapt its strategy at execution time. We accomplish this by training verifier functions using policy rollout data obtained during an initial evaluation of the policy. These verifiers are subsequently used to steer the base policy toward actions with a higher likelihood of success. Our method improves the performance of black-box diffusion policy, without changing the base parameters, making it light-weight and flexible. We present results from both simulation and real-world data and achieve an…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Robot Manipulation and Learning · Robotic Path Planning Algorithms
