Sim-to-Real Transfer and Robustness Evaluation of Reinforcement Learning Control with Integrated Perception on an ASV for Floating Waste Capture

TL;DR

This paper presents a field-validated sim-to-real transfer framework for reinforcement learning control of an autonomous surface vessel, integrating perception and robustness evaluation across diverse disturbance conditions.

Contribution

It introduces a novel two-stage simulation and perception abstraction methodology enabling reproducible field testing and explicit evaluation of the sim-to-real transfer gap for marine robots.

Findings

Achieved centimeter-level accuracy in real-world waste capture tasks.

Demonstrated robustness of the control system under 14 different disturbance regimes.

Identified actuation-model fidelity as a key factor affecting transfer performance.

Abstract

Autonomous surface vessels for floating-waste removal operate under varying hydrodynamics, external disturbances, and challenging water-surface perception. We present a field-validated system that combines camera-based polarimetric perception with a lightweight DRL-based controller for floating-waste detection and capture. Camera detections are converted into water-surface target points and tracked by a controller trained entirely in simulation and deployed directly on a retrofitted ASV platform. Our main contribution is a sim-to-real testing methodology that combines a two-stage simulation protocol with a perception abstraction module designed to mimic real camera behavior, enabling reproducible field trials and explicit evaluation of the sim-to-real gap. We apply this framework in matched simulation and field experiments across 14 disturbance regimes to expose failure modes and evaluate robustness. The results show centimeter-level terminal accuracy and indicate robust control performance under the evaluated perturbation regimes. The main source of degradation is insufficient actuation-model fidelity. We also demonstrate the system in a search-and-capture application using real camera detections in real-world conditions over areas of up to $450~m^2$. The study distills practical lessons for reliable transfer, including improved actuation-model fidelity, targeted domain randomization, and careful management of latency and timestamps across modules, while highlighting remaining challenges.