Real-Time Multi-Diver Tracking and Re-identification for Underwater Human-Robot Collaboration

TL;DR

This paper presents a real-time multi-diver tracking and re-identification system for underwater robots, combining CNN-based detection with an extended SORT algorithm to improve diver identification during human-robot collaboration.

Contribution

It introduces a novel underwater diver tracking method that integrates deep appearance features with real-time detection and tracking algorithms, tailored for autonomous robot operations.

Findings

Enhanced diver identification accuracy during tracking

Effective real-time detection and re-identification in underwater environments

Discussion of practical challenges and failure mitigation in on-board systems

Abstract

Autonomous underwater robots working with teams of human divers may need to distinguish between different divers, e.g. to recognize a lead diver or to follow a specific team member. This paper describes a technique that enables autonomous underwater robots to track divers in real time as well as to reidentify them. The approach is an extension of Simple Online Realtime Tracking (SORT) with an appearance metric (deep SORT). Initial diver detection is performed with a custom CNN designed for realtime diver detection, and appearance features are subsequently extracted for each detected diver. Next, realtime tracking-by-detection is performed with an extension of the deep SORT algorithm. We evaluate this technique on a series of videos of divers performing human-robot collaborative tasks and show that our methods result in more divers being accurately identified during tracking. We also discuss the practical considerations of applying multi-person tracking to on-board autonomous robot operations, and we consider how failure cases can be addressed during on-board tracking.