Self-Supervised Domain Mismatch Estimation for Autonomous Perception

TL;DR

This paper introduces a self-supervised autoencoder-based domain mismatch metric for autonomous perception systems, enabling monitoring of semantic segmentation performance across different domains using PSNR distributions.

Contribution

It proposes a novel domain mismatch metric based on earth mover's distance and a training-dependent threshold, improving perception reliability in autonomous driving.

Findings

DM metric correlates strongly with segmentation performance

Autoencoder-based PSNR distribution effectively monitors domain shifts

Thresholding enables functional scope definition for perception modules

Abstract

Autonomous driving requires self awareness of its perception functions. Technically spoken, this can be realized by observers, which monitor the performance indicators of various perception modules. In this work we choose, exemplarily, a semantic segmentation to be monitored, and propose an autoencoder, trained in a self-supervised fashion on the very same training data as the semantic segmentation to be monitored. While the autoencoder's image reconstruction performance (PSNR) during online inference shows already a good predictive power w.r.t. semantic segmentation performance, we propose a novel domain mismatch metric DM as the earth mover's distance between a pre-stored PSNR distribution on training (source) data, and an online-acquired PSNR distribution on any inference (target) data. We are able to show by experiments that the DM metric has a strong rank order correlation with the semantic segmentation within its functional scope. We also propose a training domain-dependent threshold for the DM metric to define this functional scope.