Robust Uncertainty Estimation for Classification of Maritime Objects

TL;DR

This paper introduces a robust uncertainty estimation method for maritime object classification, combining Monte Carlo Dropout and outlier detection, demonstrating significant improvements on CIFAR10 and a new maritime dataset.

Contribution

It presents a novel, model-agnostic uncertainty estimation approach that enhances out-of-distribution detection in maritime object classification without extensive retraining.

Findings

Improves FPR95 by 8% on CIFAR10 without OOD data

Increases performance by 77% over vanilla Wide ResNet

Enhances FPR95 by 44.2% on the SHIPS dataset

Abstract

We explore the use of uncertainty estimation in the maritime domain, showing the efficacy on toy datasets (CIFAR10) and proving it on an in-house dataset, SHIPS. We present a method joining the intra-class uncertainty achieved using Monte Carlo Dropout, with recent discoveries in the field of outlier detection, to gain more holistic uncertainty measures. We explore the relationship between the introduced uncertainty measures and examine how well they work on CIFAR10 and in a real-life setting. Our work improves the FPR95 by 8% compared to the current highest-performing work when the models are trained without out-of-distribution data. We increase the performance by 77% compared to a vanilla implementation of the Wide ResNet. We release the SHIPS dataset and show the effectiveness of our method by improving the FPR95 by 44.2% with respect to the baseline. Our approach is model agnostic, easy to implement, and often does not require model retraining.