PhononBench:A Large-Scale Phonon-Based Benchmark for Dynamical Stability in Crystal Generation

TL;DR

PhononBench introduces a large-scale benchmark for evaluating the dynamical stability of AI-generated crystal structures using phonon predictions, revealing current models' limitations and providing a valuable resource for future materials discovery.

Contribution

This work presents the first large-scale phonon-based benchmark for crystal stability, utilizing MatterSim for DFT-level accuracy and analyzing over 108,000 generated structures.

Findings

Current generative models have an average stability rate of 25.83%.

MatterGen achieves a stability rate of 41.0%.

Higher symmetry crystals tend to be more stable, with cubic systems reaching 49.2% stability.

Abstract

In this work, we introduce PhononBench, the first large-scale benchmark for dynamical stability in AI-generated crystals. Leveraging the recently developed MatterSim interatomic potential, which achieves DFT-level accuracy in phonon predictions across more than 10,000 materials, PhononBench enables efficient large-scale phonon calculations and dynamical-stability analysis for 108,843 crystal structures generated by six leading crystal generation models. PhononBench reveals a widespread limitation of current generative models in ensuring dynamical stability: the average dynamical-stability rate across all generated structures is only 25.83%, with the top-performing model, MatterGen, reaching just 41.0%. Further case studies show that in property-targeted generation-illustrated here by band-gap conditioning with MatterGen--the dynamical-stability rate remains as low as 23.5% even at the optimal band-gap condition of 0.5 eV. In space-group-controlled generation, higher-symmetry crystals exhibit better stability (e.g., cubic systems achieve rates up to 49.2%), yet the average stability across all controlled generations is still only 34.4%. An important additional outcome of this study is the identification of 28,119 crystal structures that are phonon-stable across the entire Brillouin zone, providing a substantial pool of reliable candidates for future materials exploration. By establishing the first large-scale dynamical-stability benchmark, this work systematically highlights the current limitations of crystal generation models and offers essential evaluation criteria and guidance for their future development toward the design and discovery of physically viable materials. All model-generated crystal structures, phonon calculation results, and the high-throughput evaluation workflows developed in PhononBench will be openly released at https://github.com/xqh19970407/PhononBench