# Novelty Detection via Network Saliency in Visual-based Deep Learning

## TL;DR

This paper introduces a multi-step framework for detecting novel scenarios in vision-based autonomous systems by leveraging model information and a new image similarity metric, validated on real-world and indoor racing datasets.

## Contribution

It proposes a novel approach combining model insights and a new similarity metric for novelty detection in dynamic, real-world visual data.

## Key findings

- Effective detection of novel scenarios demonstrated on real-world driving data.
- The method outperforms existing approaches in dynamic, real-world environments.
- Validated on both outdoor and indoor autonomous driving datasets.

## Abstract

Machine-learning driven safety-critical autonomous systems, such as self-driving cars, must be able to detect situations where its trained model is not able to make a trustworthy prediction. Often viewed as a black-box, it is non-obvious to determine when a model will make a safe decision and when it will make an erroneous, perhaps life-threatening one. Prior work on novelty detection deal with highly structured data and do not translate well to dynamic, real-world situations. This paper proposes a multi-step framework for the detection of novel scenarios in vision-based autonomous systems by leveraging information learned by the trained prediction model and a new image similarity metric. We demonstrate the efficacy of this method through experiments on a real-world driving dataset as well as on our in-house indoor racing environment.