FPBench

Foundation Potentials Evaluated

Exact checkpoints and versions used in this study. Model size and training-dataset size are the values used in this study's own code/records; official source links point to each architecture's primary release page.

* Pre-trained on MACE-OMAT-0, then fine-tuned on the matched MatPES functional.

† The MatterSim model card reports "Training Data Size: 6M", "Model Parameters: 4.5M", and training on "a specific variant of Density Functional Theory (PBE)". The GGA-PBE(+U) labelling, with Hubbard U applied to selected materials following Materials Project settings, is described in Yang et al. rather than on the model card. The dataset itself is not released, so the OOD label reflects the absence of documented training exposure rather than a verified composition.

‡ Rhodes et al. state that "all orb-v3-*-omat models are only trained on the AIMD subset of OMat24", and that the OMat24 dataset "contains ~55 million AIMD-sampled structures".

§ The FPBench checkpoint is nequix-oam-1. The official nequix repository documents this checkpoint as trained on OMat24, sAlex and MPtrj at the DFT (PBE+U) level. The Nequix paper describes an MPtrj-trained model and does not report a training-set size or parameter count for this released OAM checkpoint; the 707,569-parameter model size shown here was measured from the loaded checkpoint by FPBench.

‖ GPTFF’s Atomly labels were computed with VASP at the GGA-PBE level with a 520 eV plane-wave cutoff and version 5.4 PAW pseudopotentials. The functional matches the FPBench reference; the cutoff and pseudopotential version do not necessarily, so small systematic offsets relative to MatPES-PBE are expected independently of model quality.

¶ ALIGNN is trained on JARVIS-DFT at the OptB88vdW level, not PBE. Its training set is 307,113 points, split 90:5:5. It is the only FP here whose training reference functional differs from the evaluation reference, so its scores measure agreement with the FPBench MatPES-PBE reference rather than a functional-matched fitting error.

†† NEP89 is trained on eleven datasets computed at different quantum-mechanical levels rather than a single reference functional. Its FPBench MatPES-PBE scores therefore measure practical agreement with the PBE benchmark reference rather than agreement with a single matched reference functional. The eleven datasets and the 537,641-configuration total are reported in the NEP89 paper; the 976,331 trainable parameters were counted from the released model by FPBench.

‡‡ FPBench counts 42.1M parameters (42,112,223) for the GRACE potential used in inference. The released SavedModel additionally contains a 23.4M-parameter GMM uncertainty head that is disabled in FPBench runs and is therefore not included in the reported model size. The GRACE documentation does not publish a parameter count; 42.1M was measured from the released model by FPBench.

# The immediate model sources describe these labels as DFT and DFT+U total-energy labels. OMat24 itself uses PBE/PBE+U reference calculations; see the OMat24 dataset reference.

The potentials listed after the r2SCAN entries were evaluated after submission of the manuscript, which reports the ten FPs above them. Their model sizes are parameter counts measured from the loaded checkpoints.

This is the current set of FPs evaluated. As additional FPs are tested, they will be added to this table; existing benchmark results remain tied to their original model versions and are not silently replaced.

Evaluation Matrix

Which FPs were evaluated on which FPBench component/dataset. A checkmark means the FP was evaluated there; the parenthetical states whether that dataset falls inside (training) or outside (OOD, out-of-distribution) the FP's own training data, whether the FP is a fine-tune of an OOD base model onto that dataset (fine-tuned), or whether that dataset was used to pretrain the model before it was fine-tuned on other data (pretraining).

* Pre-trained on MACE-OMAT-0, fine-tuned on MatPES-PBE.

The three r2SCAN-trained FPs (M3GNet-MatPES-r2SCAN, TensorNet-MatPES-r2SCAN, MACE-MatPES-r2SCAN) are evaluated on the MatPES-r2SCAN dataset (training), in addition to the models listed above.

The "Phase Stability/Ordering & Ion Migration (NEB)" column reflects evaluation in the manuscript. Public code and results for both are now available -- see Phase Stability & Ordering and Ion Migration (NEB).