Generating Hadamard matrices with transformers

TL;DR

This paper introduces a transformer-based method combined with local search to efficiently generate Hadamard matrices, especially for larger orders where traditional methods struggle.

Contribution

The authors develop a novel transformer-augmented approach that outperforms local search in constructing large Hadamard matrices, revealing hidden symmetries in the process.

Findings

Successfully generated Hadamard matrices of order 252.

Outperformed local search methods for larger matrix orders.

Discovered and exploited hidden symmetries in the search space.

Abstract

We present a new method for constructing Hadamard matrices that combines transformer neural networks with local search in the PatternBoost framework. Our approach is designed for extremely sparse combinatorial search problems and is particularly effective for Hadamard matrices of Goethals--Seidel type, where Fourier methods permit fast scoring and optimisation. For orders between 100 and 200, it produces large numbers of inequivalent Hadamard matrices, and for larger orders, it succeeds where local search from random initialisation fails. The largest example found by our method has order 252. In addition to these new constructions, our experiments reveal that the transformer can discover and exploit useful hidden symmetry in the search space.