Towards a Solution to Bongard Problems: A Causal Approach

TL;DR

This paper introduces a causal, reinforcement learning-based approach to solving Bongard Problems, focusing on extracting meaningful representations and explanations from visual data, which are challenging for current AI methods.

Contribution

It reformulates Bongard Problems into a reinforcement learning framework and employs contrastive learning for feature extraction, enabling better interpretation and solution of these complex visual puzzles.

Findings

Reinforcement learning effectively guides decision-making in Bongard Problems.

Contrastive learning improves feature extraction from pixel data.

The combined approach enhances interpretability of learned representations.

Abstract

Even though AI has advanced rapidly in recent years displaying success in solving highly complex problems, the class of Bongard Problems (BPs) yet remain largely unsolved by modern ML techniques. In this paper, we propose a new approach in an attempt to not only solve BPs but also extract meaning out of learned representations. This includes the reformulation of the classical BP into a reinforcement learning (RL) setting which will allow the model to gain access to counterfactuals to guide its decisions but also explain its decisions. Since learning meaningful representations in BPs is an essential sub-problem, we further make use of contrastive learning for the extraction of low level features from pixel data. Several experiments have been conducted for analyzing the general BP-RL setup, feature extraction methods and using the best combination for the feature space analysis and its interpretation.