DIO: Refining Mutual Information and Causal Chain to Enhance Machine Abstract Reasoning Ability

TL;DR

This paper introduces the DIO framework with three key refinements to improve machine abstract reasoning on Raven's Progressive Matrices by embedding causal logic and tightening mutual information bounds.

Contribution

It proposes novel methods—Brando, WORLD, and DIEGO—to enhance the causal modeling and bound tightness in DIO, significantly improving RPM performance and answer generation.

Findings

DIO achieves higher accuracy on RPM benchmarks.

Refinements enable DIO to generate valid open-ended answers.

The approach offers causal-driven design insights for reasoning models.

Abstract

Despite deep learning's broad success, its abstract-reasoning bottleneck persists. We tackle Raven's Progressive Matrices (RPM), the benchmark for pattern, reasoning and problem-solving intelligence. We model the full causal chain image $\rightarrow$ attributes $\rightarrow$ progressive patterns $\rightarrow$ consistency $\rightarrow$ answer and build the baseline DIO. Yet DIO's mutual-information lower-bound objective does not embed human logic: the bound is loose and statistic-based, ignoring causal subject-object links. We therefore present three refinements. 1) Brando introduces trainable negative options to tighten the variational bound. 2) WORLD replaces generation with a Gaussian-mixture feature model that supplies infinite, weighted negatives, further tightening the bound. 3) DIEGO adds metadata supervision to rectify the "attributes $\rightarrow$ patterns" semantic gap, aligning representations with human rules. These upgrades substantially boost discriminative RPM accuracy and, for the first time, let DIO generate valid answers in open-ended RPM. The work provides causal-driven design guidelines, objective-refinement strategies and cross-modal insights for abstract-reasoning research.