Stochastic Grounded Action Transformation for Robot Learning in Simulation

TL;DR

This paper introduces SGAT, an extension of GAT, that models environmental stochasticity to improve the transfer of robot control policies from simulation to real-world settings.

Contribution

The paper proposes SGAT, a novel algorithm that explicitly accounts for stochasticity in the environment, enhancing sim-to-real transfer robustness.

Findings

SGAT outperforms GAT in stochastic environments.

SGAT produces more robust policies in real-world robot tasks.

Experimental results confirm SGAT's effectiveness in simulation and physical domains.

Abstract

Robot control policies learned in simulation do not often transfer well to the real world. Many existing solutions to this sim-to-real problem, such as the Grounded Action Transformation (GAT) algorithm, seek to correct for or ground these differences by matching the simulator to the real world. However, the efficacy of these approaches is limited if they do not explicitly account for stochasticity in the target environment. In this work, we analyze the problems associated with grounding a deterministic simulator in a stochastic real world environment, and we present examples where GAT fails to transfer a good policy due to stochastic transitions in the target domain. In response, we introduce the Stochastic Grounded Action Transformation(SGAT) algorithm,which models this stochasticity when grounding the simulator. We find experimentally for both simulated and physical target domains that SGAT can find policies that are robust to stochasticity in the target domain