TACIT: Transformation-Aware Capturing of Implicit Thought

TL;DR

TACIT introduces a diffusion-based transformer that visualizes implicit reasoning in pixel space, demonstrating rapid, holistic maze-solving with insights into neural reasoning processes.

Contribution

The paper presents a novel pixel-space diffusion transformer enabling interpretable visual reasoning and revealing holistic, insight-like solution emergence.

Findings

192x reduction in training loss over 100 epochs

22.7x improvement in L2 distance to ground truth

Solution emergence within 2% of the transformation process

Abstract

We present TACIT (Transformation-Aware Capturing of Implicit Thought), a diffusion-based transformer for interpretable visual reasoning. Unlike language-based reasoning systems, TACIT operates entirely in pixel space using rectified flow, enabling direct visualization of the reasoning process at each inference step. We demonstrate the approach on maze-solving, where the model learns to transform images of unsolved mazes into solutions. Key results on 1 million synthetic maze pairs include: - 192x reduction in training loss over 100 epochs - 22.7x improvement in L2 distance to ground truth - Only 10 Euler steps required (vs. 100-1000 for typical diffusion models) Quantitative analysis reveals a striking phase transition phenomenon: the solution remains invisible for 68% of the transformation (zero recall), then emerges abruptly at t=0.70 within just 2% of the process. Most remarkably, 100% of samples exhibit simultaneous emergence across all spatial regions, ruling out sequential path construction and providing evidence for holistic rather than algorithmic reasoning. This "eureka moment" pattern -- long incubation followed by sudden crystallization -- parallels insight phenomena in human cognition. The pixel-space design with noise-free flow matching provides a foundation for understanding how neural networks develop implicit reasoning strategies that operate below and before language.