A Chemically Grounded Evaluation Framework for Generative Models in Materials Discovery

TL;DR

This paper introduces a chemically grounded evaluation framework for generative models in materials discovery, combining DFT-based stability analysis with machine learning metrics to improve assessment accuracy.

Contribution

It presents a new evaluation framework that integrates DFT simulations with ML metrics, addressing limitations of conventional metrics in assessing chemical feasibility.

Findings

Conventional ML metrics can misrepresent chemical feasibility.

Simulation-informed evaluation improves reliability of generative models.

Proposes robust metrics for chemical stability assessment.

Abstract

Generative models hold great promise for accelerating materials discovery, but their evaluation often overlooks the chemical validity and stability requirements crucial to real-world applications. Density Functional Theory (DFT) simulations are the gold standard for evaluating such properties but are computationally intensive and inaccessible to non-experts. We propose a chemically grounded, user-friendly evaluation framework that integrates DFT-based stability analysis with commonly used machine learning (ML) metrics. Through systematic experiments using both perturbative and generative methods, we demonstrate that conventional ML metrics can misrepresent chemical feasibility. To address this, we propose new insights on robust metrics and highlight the importance of simulation-informed evaluation for developing reliable generative models in materials science.