Phononbench: A large-scale benchmark reveals the dynamical stability challenge in AI-generated crystals

The Dynamical Stability Challenge in AI-Generated Crystals

Artificial intelligence has rapidly transformed crystal materials design. Graph neural networks, diffusion models, flow-matching approaches, and large language models can now generate large numbers of novel crystal structures, providing new opportunities for exploring the vast chemical and structural space of materials. However, current evaluations of AI-generated crystals mainly focus on conventional criteria such as stability, uniqueness, and novelty (S.U.N.), with stability often assessed through thermodynamic quantities such as energy above the convex hull. Thermodynamic stability and dynamical stability describe different physical aspects of a crystal. While thermodynamic stability reflects the energetic tendency of a structure against decomposition, dynamical stability concerns whether the structure resides in a local potential-energy minimum and can withstand small perturbations. Dynamical instability is commonly manifested through imaginary phonon modes. Yet full phonon calculations are computationally demanding, making it difficult to systematically evaluate the dynamical stability of tens or hundreds of thousands of AI-generated structures. As a result, many generated crystals that appear structurally or thermodynamically plausible have not been systematically tested for dynamical stability.

To address this challenge, the researchers introduced PhononBench, a large-scale benchmark designed specifically to bring phonon-based dynamical-stability evaluation into the assessment of AI-generated crystals. The study covered seven representative crystal-generation approaches, including GNN- and diffusion-based models, flow-matching approaches, space-group-based generation methods, and large language model-based crystal generation. A total of 252,437 novel crystal structures were generated. After deduplication, CIF validation, post-processing, and structural relaxation, 133,838 crystals were subjected to phonon-spectrum calculations using MatterSim-v1 combined with Phonopy. The benchmark reveals that dynamical stability remains a substantial challenge for current AI crystal-generation models. On the MP20 dataset, MatterGen, DiffCSP, and CrystalFlow achieve dynamical-stability rates of 45.05%, 43.94%, and 40.00%, respectively. LLM/Transformer-based approaches show lower stability overall, while optimization through prompting and sampling enables LLaMA2-70B to reach 32.50%.

On the Alex20 dataset, the overall stability rates are higher, with MatterGen and InvDesFlow-AL reaching 57.89% and 56.59%, respectively. Across the eight independent MP20 entries, the unweighted mean dynamical-stability rate is 30.7%, with MatterGen reaching 45.0%.These results highlight that, despite the rapid progress of AI crystal-generation models, generating structures that are dynamically stable remains a major challenge. An important additional outcome of PhononBench is the identification of 32,995 crystal structures whose minimum phonon frequencies are above −0.001 THz along the sampled high-symmetry paths. These structures provide a large pool of MatterSim-labeled dynamically stable AI-generated crystals for subsequent materials research and stability-aware model development.

Why Are AI-Generated Crystals Dynamically Unstable?

The researchers further investigated the physical origins of dynamical instability from multiple perspectives, including chemical composition, chemical complexity, local coordination, and electronic characteristics. The results show that dynamical instability is not randomly distributed across the generated structures. Instead, it exhibits clear chemical and structural trends.Structures containing highly electronegative elements such as F, O, Cl, H, Br, and I tend to exhibit higher probabilities of dynamical instability. The study also finds that the probability of instability increases with the number of constituent elements. As chemical complexity increases, satisfying local bonding preferences, coordination requirements, charge balance, and geometric constraints simultaneously becomes more difficult, making local distortions and lattice instabilities more likely. Local coordination also plays an important role. The instability probability decreases from approximately 84% in low-coordination structures to about 38% in high-coordination structures. This suggests that under-coordinated structures, which lack sufficient local geometric constraints, are more susceptible to soft phonon modes, whereas higher coordination can provide stronger local structural confinement. The researchers also examined the relationship between electronic characteristics and dynamical instability. Metallic and insulating structures exhibit different instability behaviors, particularly for weak instabilities associated with subtle structural distortions. Importantly, the study shows that thermodynamic stability does not directly guarantee dynamical stability. A crystal can satisfy conventional energetic stability criteria while still exhibiting imaginary phonon modes.These findings suggest that AI crystal-generation models need to learn more than simply producing structures that are chemically plausible or thermodynamically favorable. They also need to better capture the complex bonding constraints, local structural environments, and lattice-vibrational characteristics that govern dynamical stability.

The Key Enabler for High-Throughput Phonon Calculations

Systematically evaluating more than 100,000 AI-generated crystals using conventional first-principles phonon calculations would require substantial computational resources. To overcome this bottleneck, the researchers developed a high-throughput phonon-calculation workflow based on MatterSim-v1 and Phonopy. MatterSim-v1 provides an efficient machine-learning interatomic potential for calculating energies and forces, while Phonopy is used to construct force constants and obtain phonon spectra. This combination makes it possible to perform phonon calculations at a scale that would be difficult to achieve using conventional DFT workflows alone. In the benchmark comparison, MatterSim demonstrates strong performance in dynamical-stability classification. On the benchmark dataset containing 8,189 dynamically stable and 1,769 dynamically unstable structures, MatterSim achieves a true-stable rate of approximately 95%, with about 5% of stable structures classified as unstable. Its true-unstable rate reaches approximately 75%. On a representative subset of 100 structures, including 50 stable and 50 unstable systems, MatterSim identifies approximately 96% of the stable structures, compared with 100% in the DFT reference. At the same time, its substantially higher computational efficiency enables phonon calculations to be extended to more than 100,000 structures. Compared with other recently developed universal machine-learning interatomic potentials, MatterSim provides a favorable balance between phonon-stability prediction performance and computational efficiency. This balance forms an important computational foundation for the large-scale analysis performed in PhononBench.

[1]Antoine Loew, Dewen Sun, Hai-Chen Wang, Silvana Botti, and Miguel A. L. Marques. Universal machine learning interatomic potentials are ready for phonons. npj Computational Materials,

11(1):178, 2025

Since the launch of the PhononBench website, the platform has provided more than 24,000 service calls to researchers. By combining a universal machine-learning interatomic potential with the Phonopy workflow, the platform reduces conventional high-cost phonon calculations to a minute-level timescale, substantially lowering the barrier to dynamical-stability verification. The platform is designed with a simple workflow. Users can upload a CIF file and directly submit a calculation, or access advanced settings according to their research needs. The platform supports both 3D and 2D materials, customizable supercell parameters, NPT/NVT optimization modes, and crystal-symmetry constraints. For 2D materials, vacuum-layer treatment and a 4 × 4 × 1 supercell are automatically applied, while 3D crystals use a default 2 × 2 × 2 supercell. After submission, the platform automatically performs structural relaxation, force-constant calculation, and phonon-spectrum analysis, and returns the corresponding results without requiring users to manually configure a complex computational workflow.

The Future: Toward Stability-Aware AI Materials Discovery

PhononBench demonstrates that dynamical stability should become an important component of the evaluation and development of AI-generated materials. Beyond identifying dynamically stable structures, the benchmark provides phonon-labeled crystal structures and stable–unstable structure pairs that can support future stability-aware generation and optimization strategies. These data may enable generative models to explicitly learn dynamical-stability preferences rather than relying solely on structural or thermodynamic information. At the same time, the stability labels reported by PhononBench should be interpreted as MatterSim-self-consistent labels under the specified computational protocol, rather than exact DFT-level stability assignments. For selected candidate materials, independent first-principles structural relaxation and phonon calculations remain important for further validation. By combining a large-scale benchmark, a substantial phonon-labeled crystal dataset, and an accessible online calculation platform, PhononBench provides a practical framework for moving AI-driven materials discovery beyond structural plausibility and thermodynamic stability toward more comprehensive physical evaluation.

Reference

Xiao-Qi Han, Ze-Feng Gao, Wen-Kao Li, Peng-Jie Guo, Zhong-Yi Lu. PhononBench:A Large-Scale Phonon-Based Benchmark for Dynamical Stability in Crystal Generation[J]. AI for Science. DOI: 10.1088/3050-287X/ae9ee4