SeePerSea: Multi-modal Perception Dataset of In-water Objects for Autonomous Surface Vehicles

TL;DR

This paper presents SeePerSea, a comprehensive multi-modal perception dataset for in-water object detection in autonomous maritime navigation, enabling improved situational awareness for Autonomous Surface Vehicles (ASVs).

Contribution

It introduces the first publicly available labeled multi-modal dataset for in-water objects, collected over four years, to advance perception algorithms in marine environments.

Findings

Deep learning models trained on the dataset show promising results.

The dataset reveals challenges in in-water perception under varying conditions.

Open research directions are discussed based on the dataset's analysis.

Abstract

This paper introduces the first publicly accessible labeled multi-modal perception dataset for autonomous maritime navigation, focusing on in-water obstacles within the aquatic environment to enhance situational awareness for Autonomous Surface Vehicles (ASVs). This dataset, collected over 4 years and consisting of diverse objects encountered under varying environmental conditions, aims to bridge the research gap in ASVs by providing a multi-modal, annotated, and ego-centric perception dataset, for object detection and classification. We also show the applicability of the proposed dataset by training and testing current deep learning-based open-source perception algorithms that have shown success in the autonomous ground vehicle domain. With the training and testing results, we discuss open challenges for existing datasets and methods, identifying future research directions. We expect that our dataset will contribute to the development of future marine autonomy pipelines and marine (field) robotics. This dataset is open source and found at https://seepersea.github.io/.