Abduction of Domain Relationships from Data for VQA

TL;DR

This paper introduces a method to infer domain relationships from data to improve visual question answering (VQA) systems that operate with logic-based representations, enhancing accuracy with minimal examples.

Contribution

It presents a novel abduction-based approach to derive domain relationships in VQA, complementing existing knowledge augmentation techniques.

Findings

Significant accuracy improvement in VQA tasks.

Effective with few training examples.

Baseline approach demonstrates practical feasibility.

Abstract

In this paper, we study the problem of visual question answering (VQA) where the image and query are represented by ASP programs that lack domain data. We provide an approach that is orthogonal and complementary to existing knowledge augmentation techniques where we abduce domain relationships of image constructs from past examples. After framing the abduction problem, we provide a baseline approach, and an implementation that significantly improves the accuracy of query answering yet requires few examples.