Plants Don't Walk on the Street: Common-Sense Reasoning for Reliable Semantic Segmentation

TL;DR

This paper introduces a method that combines human-designed rules and learned knowledge within a probabilistic logic framework to enhance the robustness of semantic segmentation in autonomous driving, especially under severe image distortions.

Contribution

It presents a novel integration of common-sense knowledge using Probabilistic Soft Logic to improve the reliability of deep neural networks in semantic segmentation tasks.

Findings

Improved robustness against image distortions

Enhanced semantic segmentation accuracy

Effective integration of common knowledge into neural networks

Abstract

Data-driven sensor interpretation in autonomous driving can lead to highly implausible predictions as can most of the time be verified with common-sense knowledge. However, learning common knowledge only from data is hard and approaches for knowledge integration are an active research area. We propose to use a partly human-designed, partly learned set of rules to describe relations between objects of a traffic scene on a high level of abstraction. In doing so, we improve and robustify existing deep neural networks consuming low-level sensor information. We present an initial study adapting the well-established Probabilistic Soft Logic (PSL) framework to validate and improve on the problem of semantic segmentation. We describe in detail how we integrate common knowledge into the segmentation pipeline using PSL and verify our approach in a set of experiments demonstrating the increase in robustness against several severe image distortions applied to the A2D2 autonomous driving data set.