# Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentation

## TL;DR

This paper introduces an IMU-assisted semantic segmentation algorithm for obstacle detection on unmanned surface vehicles, improving accuracy and reducing false detections by integrating horizon estimation and stereo verification.

## Contribution

It extends semantic segmentation with IMU data for horizon estimation, proposes a stereo verification method, and provides a new multi-modal dataset for USV obstacle detection.

## Key findings

- 30% improvement in water-edge detection accuracy
- Over 21% reduction in false positive rate
- Almost 60% reduction in false negative rate

## Abstract

A new obstacle detection algorithm for unmanned surface vehicles (USVs) is presented. A state-of-the-art graphical model for semantic segmentation is extended to incorporate boat pitch and roll measurements from the on-board inertial measurement unit (IMU), and a stereo verification algorithm that consolidates tentative detections obtained from the segmentation is proposed. The IMU readings are used to estimate the location of horizon line in the image, which automatically adjusts the priors in the probabilistic semantic segmentation model. We derive the equations for projecting the horizon into images, propose an efficient optimization algorithm for the extended graphical model, and offer a practical IMU-camera-USV calibration procedure. Using an USV equipped with multiple synchronized sensors, we captured a new challenging multi-modal dataset, and annotated its images with water edge and obstacles. Experimental results show that the proposed algorithm significantly outperforms the state of the art, with nearly 30% improvement in water-edge detection accuracy, an over 21% reduction of false positive rate, an almost 60% reduction of false negative rate, and an over 65% increase of true positive rate, while its Matlab implementation runs in real-time.