False Negative Reduction in Video Instance Segmentation using Uncertainty Estimates

TL;DR

This paper introduces a post-processing method for video instance segmentation that reduces false negatives by analyzing temporal inconsistencies and uncertainty estimates, improving detection accuracy in applications like automated driving.

Contribution

The paper presents a novel false negative detection approach using temporal inconsistencies and uncertainty estimates, applicable as a post-processing step for any neural network trained on single frames.

Findings

Improved false negative and false positive trade-off in video segmentation.

Effective false negative detection based on temporal inconsistencies.

Uncertainty-based pruning enhances overall detection performance.

Abstract

Instance segmentation of images is an important tool for automated scene understanding. Neural networks are usually trained to optimize their overall performance in terms of accuracy. Meanwhile, in applications such as automated driving, an overlooked pedestrian seems more harmful than a falsely detected one. In this work, we present a false negative detection method for image sequences based on inconsistencies in time series of tracked instances given the availability of image sequences in online applications. As the number of instances can be greatly increased by this algorithm, we apply a false positive pruning using uncertainty estimates aggregated over instances. To this end, instance-wise metrics are constructed which characterize uncertainty and geometry of a given instance or are predicated on depth estimation. The proposed method serves as a post-processing step applicable to any neural network that can also be trained on single frames only. In our tests, we obtain an improved trade-off between false negative and false positive instances by our fused detection approach in comparison to the use of an ordinary score value provided by the instance segmentation network during inference.