Sim and Real: Better Together
Shirli Di Castro Shashua, Dotan Di Castro, Shie Mannor

TL;DR
This paper introduces a method for simultaneous learning from simulation and real-world interactions in robotic manipulation, balancing sample efficiency and fidelity to improve real-world performance.
Contribution
The paper proposes a novel algorithm with a theoretical analysis for balancing simulation and real data in reinforcement learning, enhancing sim-to-real transfer.
Findings
Effective in sim-to-real transfer tasks
Theoretical convergence guarantees provided
Improved sample efficiency in real environment
Abstract
Simulation is used extensively in autonomous systems, particularly in robotic manipulation. By far, the most common approach is to train a controller in simulation, and then use it as an initial starting point for the real system. We demonstrate how to learn simultaneously from both simulation and interaction with the real environment. We propose an algorithm for balancing the large number of samples from the high throughput but less accurate simulation and the low-throughput, high-fidelity and costly samples from the real environment. We achieve that by maintaining a replay buffer for each environment the agent interacts with. We analyze such multi-environment interaction theoretically, and provide convergence properties, through a novel theoretical replay buffer analysis. We demonstrate the efficacy of our method on a sim-to-real environment.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsSimulation Techniques and Applications · Parallel Computing and Optimization Techniques · Scientific Computing and Data Management
