Visual Explanation by High-Level Abduction: On Answer-Set Programming Driven Reasoning about Moving Objects

TL;DR

This paper introduces a hybrid system combining answer set programming and visual processing to generate robust explanations of moving objects in videos, integrating hypothesis formation, belief revision, and default reasoning.

Contribution

It presents a formal framework and implementation for abductive reasoning about moving objects in videos using answer set programming integrated with visual data analysis.

Findings

Effective reasoning on diverse video datasets

Robust visual explanations generated from complex data

Formal declarative method demonstrated on benchmarks

Abstract

We propose a hybrid architecture for systematically computing robust visual explanation(s) encompassing hypothesis formation, belief revision, and default reasoning with video data. The architecture consists of two tightly integrated synergistic components: (1) (functional) answer set programming based abductive reasoning with space-time tracklets as native entities; and (2) a visual processing pipeline for detection based object tracking and motion analysis. We present the formal framework, its general implementation as a (declarative) method in answer set programming, and an example application and evaluation based on two diverse video datasets: the MOTChallenge benchmark developed by the vision community, and a recently developed Movie Dataset.