# Iterative Inference in a Chess-Playing Neural Network

## TL;DR

This paper investigates how a chess neural network refines its representations across layers, revealing complex computational phases and late-layer reversals influenced by safety considerations.

## Contribution

It extends the logit lens analysis to a chess engine, uncovering detailed insights into the layered computational processes and heuristic influences.

## Key findings

- Capability improves across layers but in distinct phases
- Move preferences are reevaluated throughout the network
- Final layers prioritize safety over aggression

## Abstract

Do neural networks build their representations through smooth, gradual refinement, or via more complex computational processes? We investigate this by extending the logit lens to analyze the policy network of Leela Chess Zero, a superhuman chess engine. Although playing strength and puzzle-solving ability improve consistently across layers, capability progression occurs in distinct computational phases with move preferences undergoing continuous reevaluation--move rankings remain poorly correlated with final outputs until late, and correct puzzle solutions found in middle layers are sometimes overridden. This late-layer reversal is accompanied by concept preference analyses showing final layers prioritize safety over aggression, suggesting a mechanism by which heuristic priors can override tactical solutions.