Transformer-Enhanced Variational Autoencoder for Crystal Structure Prediction

TL;DR

This paper introduces TransVAE-CSP, a novel transformer-enhanced variational autoencoder that effectively captures periodicity and symmetry in crystal structures, improving prediction and generation accuracy for materials science applications.

Contribution

It presents a new model integrating adaptive distance expansion and equivariant transformer encoding to better learn crystal structure distributions.

Findings

Outperforms existing methods in structure reconstruction.

Achieves higher accuracy in crystal structure generation.

Demonstrates effectiveness across multiple datasets.

Abstract

Crystal structure forms the foundation for understanding the physical and chemical properties of materials. Generative models have emerged as a new paradigm in crystal structure prediction(CSP), however, accurately capturing key characteristics of crystal structures, such as periodicity and symmetry, remains a significant challenge. In this paper, we propose a Transformer-Enhanced Variational Autoencoder for Crystal Structure Prediction (TransVAE-CSP), who learns the characteristic distribution space of stable materials, enabling both the reconstruction and generation of crystal structures. TransVAE-CSP integrates adaptive distance expansion with irreducible representation to effectively capture the periodicity and symmetry of crystal structures, and the encoder is a transformer network based on an equivariant dot product attention mechanism. Experimental results on the carbon_24, perov_5, and mp_20 datasets demonstrate that TransVAE-CSP outperforms existing methods in structure reconstruction and generation tasks under various modeling metrics, offering a powerful tool for crystal structure design and optimization.