Materium: An Autoregressive Approach for Material Generation

TL;DR

Materium introduces an autoregressive transformer for rapid, scalable crystal structure generation by converting 3D representations into token sequences, outperforming diffusion methods in speed and efficiency.

Contribution

The paper presents a novel autoregressive transformer model for crystal generation that directly predicts atomic positions, enabling faster and scalable material design compared to existing diffusion-based approaches.

Findings

Materium generates crystal structures efficiently with high accuracy.

The model performs well across multiple property conditions.

Generation speed surpasses diffusion-based methods.

Abstract

We present Materium: an autoregressive transformer for generating crystal structures that converts 3D material representations into token sequences. These sequences include elements with oxidation states, fractional coordinates and lattice parameters. Unlike diffusion approaches, which refine atomic positions iteratively through many denoising steps, Materium places atoms at precise fractional coordinates, enabling fast, scalable generation. With this design, the model can be trained in a few hours on a single GPU and generate samples much faster on GPUs and CPUs than diffusion-based approaches. The model was trained and evaluated using multiple properties as conditions, including fundamental properties, such as density and space group, as well as more practical targets, such as band gap and magnetic density. In both single and combined conditions, the model performs consistently well, producing candidates that align with the requested inputs.