Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objects

TL;DR

This paper introduces KGFP, a novel framework that detects when object detectors fail silently in safety-critical environments by measuring semantic misalignment between detector features and foundation model embeddings, improving failure detection accuracy.

Contribution

The paper presents KGFP, a new representation-based monitoring method that effectively predicts detector failures by leveraging semantic divergence, outperforming traditional OOD detection approaches.

Findings

KGFP improves safety-critical object detection by reliably flagging failures.

Applying KGFP increases recall of critical objects at low false positive rates.

KGFP outperforms baseline OOD methods across multiple visual domains.

Abstract

Object detectors deployed in safety-critical environments can fail silently, e.g. missing pedestrians, workers, or other safety-critical objects without emitting any warning. Traditional Out Of Distribution (OOD) detection methods focus on identifying unfamiliar inputs, but do not directly predict functional failures of the detector itself. We introduce Knowledge Guided Failure Prediction (KGFP), a representation-based monitoring framework that treats missed safety-critical detections as anomalies to be detected at runtime. KGFP measures semantic misalignment between internal object detector features and visual foundation model embeddings using a dual-encoder architecture with an angular distance metric. A key property is that when either the detector is operating outside its competence or the visual foundation model itself encounters novel inputs, the two embeddings diverge, producing a high-angle signal that reliably flags unsafe images. We compare our novel KGFS method to baseline OOD detection methods. On COCO person detection, applying KGFP as a selective-prediction gate raises person recall among accepted images from 64.3% to 84.5% at 5% False Positive Rate (FPR), and maintains strong performance across six COCO-O visual domains, outperforming OOD baselines by large margins. Our code, models, and features are published at https://gitlab.cc-asp.fraunhofer.de/iosb_public/KGFP.