Show Why the Answer is Correct! Towards Explainable AI using Compositional Temporal Attention

TL;DR

This paper introduces a dynamic neural network with compositional temporal attention for Visual Question Answering, enhancing reasoning and interpretability, especially for complex questions, and demonstrating superior performance on benchmark datasets.

Contribution

It proposes a novel dynamic neural network architecture that dynamically assembles modules and uses compositional temporal attention to improve reasoning and interpretability in VQA.

Findings

Outperforms previous methods on VQA2.0 and CLEVR datasets.

Provides better reasoning explanations for answers.

Achieves improved understanding of complex questions.

Abstract

Visual Question Answering (VQA) models have achieved significant success in recent times. Despite the success of VQA models, they are mostly black-box models providing no reasoning about the predicted answer, thus raising questions for their applicability in safety-critical such as autonomous systems and cyber-security. Current state of the art fail to better complex questions and thus are unable to exploit compositionality. To minimize the black-box effect of these models and also to make them better exploit compositionality, we propose a Dynamic Neural Network (DMN), which can understand a particular question and then dynamically assemble various relatively shallow deep learning modules from a pool of modules to form a network. We incorporate compositional temporal attention to these deep learning based modules to increase compositionality exploitation. This results in achieving better understanding of complex questions and also provides reasoning as to why the module predicts a particular answer. Experimental analysis on the two benchmark datasets, VQA2.0 and CLEVR, depicts that our model outperforms the previous approaches for Visual Question Answering task as well as provides better reasoning, thus making it reliable for mission critical applications like safety and security.