A generative material transformer using Wyckoff representation

TL;DR

This paper introduces Matra-Genoa, a transformer-based generative model for crystal structures that efficiently produces stable, novel compounds by leveraging symmetry-aware representations and conditioning on stability metrics.

Contribution

The work presents a new autoregressive transformer model that incorporates symmetry and property conditioning, significantly improving the stability and diversity of generated crystal structures.

Findings

Generated structures are 8 times more likely to be stable than baselines.

Model can generate 3 million unique crystals, including verified stable compounds.

Efficiently samples from a hybrid action space across the periodic table.

Abstract

Materials play a critical role in various technological applications. Identifying and enumerating stable compounds, those near the convex hull, is therefore essential. Despite recent progress, generative models either have a relatively low rate of stable compounds, are computationally expensive, or lack symmetry. In this work we present Matra-Genoa, an autoregressive transformer model built on invertible tokenized representations of symmetrized crystals, including free coordinates. This approach enables sampling from a hybrid action space. The model is trained across the periodic table and space groups and can be conditioned on specific properties. We demonstrate its ability to generate stable, novel, and unique crystal structures by conditioning on the distance to the convex hull. Resulting structures are 8 times more likely to be stable than baselines using PyXtal with charge compensation, while maintaining high computational efficiency. We also release a dataset of 3 million unique crystals generated by our method, including 4,000 compounds verified by density-functional theory to be within 0.001 eV/atom of the convex hull.