Application-oriented force-error metrics for foundation potentials (FPs), evaluated on MatPES-PBE, MatPES-r2SCAN, and OMat24 rattled-1000.
The summary leaderboard highlights average force error together with application-oriented metrics for highly accurate force predictions, large-force-error atoms, and far-from-equilibrium (FE) atoms. Highly accurate force predictions have force-magnitude errors |Δ|F|| < 0.01 eV/Å, large-force-error atoms have |Δ|F|| > 1 eV/Å, and FE atoms are defined by |FDFT| > 1 eV/Å. Full definitions are in Metrics, below the leaderboard.
The per-column best value (green) updates when you click a column header. FP badges show whether that FP's own training data included this dataset's functional/source (training), included it only during pretraining before the released checkpoint was fine-tuned onto a different dataset (pretraining), did not include it (OOD), or is a fine-tune of an OOD base model onto this dataset (fine-tuned).
| FP | Average errorΔ|F|MAE / RMSE(eV/Å) ↓ | Large-force-error atomsFrac(|Δ|F|| > 1 eV/Å)(%) ↓ | FE atomsΔ|F|MAE / RMSE(eV/Å) ↓ | FE atomsΔθMAE / RMSE(°) ↓ | Highly accurate force predictionsFrac(|Δ|F|| < 0.01 eV/Å)(%) ↑ | Joint force magnitude-angle accuracyFrac(<0.01 eV/Å& <1°/20°)(%) ↑ | Average errorΔθMAE / RMSE(°) ↓ | Excluding large-error atomsΔ|F|MAE / RMSE(eV/Å), on |Δ|F|| < 1 eV/Å ↓ | Version (training) |
|---|---|---|---|---|---|---|---|---|---|
| Orb OOD | 0.17 / 1.12 | 2.00 | 0.31 / 2.09 | 12 / 24 | 12.6 | 1.4 / 8.9 | 30 / 52 | 0.14 / 0.22 | orb-v3-conservative-inf-omat-20250404 (OMat24) |
| SevenNet OOD | 0.19 / 0.91 | 2.63 | 0.39 / 1.70 | 15 / 28 | 11.2 | 1.6 / 7.4 | 33 / 55 | 0.15 / 0.24 | 7net-mf-ompa (modal mpa) (OMat24 + sAlex + MPtrj) |
| MatterSim OOD | 0.21 / 1.29 | 2.65 | 0.40 / 2.41 | 16 / 28 | 7.9 | 1.00 / 4.5 | 37 / 59 | 0.17 / 0.26 | MatterSim-v1.0.0-5M (MatterSim dataset (PBE)) |
| Nequix OOD | 0.21 / 1.52 | 2.83 | 0.43 / 2.86 | 15 / 27 | 10.7 | 1.6 / 6.9 | 33 / 55 | 0.16 / 0.25 | nequix-oam-1 (OMat24 + sAlex + MPtrj) |
| MACE OOD | 0.23 / 1.93 | 3.44 | 0.49 / 3.65 | 18 / 29 | 8.34 | 1.07 / 4.51 | 37 / 58 | 0.17 / 0.27 | >=v0.3.10 (MACE-MPA-0, medium) (MPtrj + sAlex) |
| CHGNet OOD | 0.31 / 2.19 | 5.71 | 0.70 / 4.14 | 24 / 36 | 5.61 | 0.66 / 2.16 | 45 / 66 | 0.22 / 0.32 | v0.3.0 (MPtrj) |
| GPTFF OOD | 0.42 / 1.89 | 9.82 | 1.00 / 3.55 | 30 / 43 | 4.7 | 0.59 / 1.4 | 50 / 70 | 0.26 / 0.36 | gptff_v2 (Atomly (PBE)) |
| M3GNet OOD | 0.44 / 1.49 | 10.79 | 0.98 / 2.76 | 40 / 56 | 4.56 | 0.54 / 1.45 | 56 / 76 | 0.26 / 0.36 | MP-2021.2.8-PES (MP-2021.2.8) |
| UMA OOD | 0.56 / 7.93 | 2.69 | 1.13 / 11.89 | 12 / 25 | 15.13 | 2.79 / 11.44 | 28 / 51 | 0.12 / 0.21 | s-1p1 (OC20+ |
| ALIGNN OOD functional mismatch | 2.08 / 24.80 | 24.23 | 5.51 / 46.95 | 61 / 77 | 2.5 | 0.20 / 0.55 | 70 / 87 | 0.31 / 0.40 | alignnff_wt10 (JARVIS-DFT)OptB88vdW → PBE |
| M3GNet-MatPES training | 0.21 / 1.24 | 2.35 | 0.43 / 2.34 | 18 / 27 | 7.14 | 0.92 / 3.75 | 34 / 53 | 0.17 / 0.25 | v2025.1 (MatPES-PBE) |
| TensorNet-MatPES training | 0.17 / 1.39 | 1.20 | 0.33 / 2.63 | 12 / 18 | 7.66 | 0.98 / 4.65 | 29 / 49 | 0.15 / 0.22 | v2025.1 (MatPES-PBE) |
| MACE-MatPES training | 0.08 / 1.10 | 0.25 | 0.16 / 2.09 | 6 / 10 | 19.02 | 3.88 / 15.35 | 16 / 32 | 0.07 / 0.13 | >=v0.3.10 (MatPES-PBE (fine-tuned)) |
| NEP89 OOD | 0.40 / 3.70 | 7.60 | 0.74 / 6.99 | 31 / 44 | 3.2 | 0.31 / 1.3 | 52 / 73 | 0.27 / 0.36 | nep89_20250409 (mixed QM levels) |
| DPA4 OOD | 0.15 / 2.16 | 1.64 | 0.27 / 3.71 | 10 / 22 | 19.7 | 4.7 / 16 | 25 / 48 | 0.11 / 0.20 | DPA4-Plus-OMat24-v20260805 (OMat24) |
| GRACE OOD | 0.16 / 1.49 | 2.08 | 0.30 / 2.81 | 12 / 25 | 17.8 | 3.7 / 14 | 27 / 50 | 0.12 / 0.21 | GRACE-3L-OMAT-large-ft-AM (OMat24 → sAlex + MPtrj) |
| eqV2 OOD | 0.16 / 1.27 | 1.98 | 0.28 / 2.39 | 11 / 25 | 14.6 | 2.5 / 12 | 28 / 51 | 0.12 / 0.21 | eqV2_31M_omat_mp_salex (OMat24 → MPtrj + sAlex) |
| eSEN OOD | 0.15 / 1.38 | 1.93 | 0.28 / 2.60 | 11 / 25 | 18.6 | 4.0 / 15 | 27 / 51 | 0.12 / 0.21 | esen_30m_oam (OMat24 → MPtrj + sAlex) |
Hover or click the model markers (* † ‡ §) in the table for model-specific details.
* MatterSim’s training dataset is not publicly documented in detail. The model card states only that the released MatterSim-v1.0.0-5M checkpoint was trained on a PBE-based dataset of approximately 6M structures; the dataset is neither named nor released. Its OOD label therefore reflects the absence of any documented MatPES or OMat24 training exposure, not a verified dataset composition.
† The FPBench checkpoint is nequix-oam-1. The official nequix repository documents this checkpoint as trained on OMat24, sAlex and MPtrj at the DFT (PBE+U) level. The Nequix paper (Koker, Kotak & Smidt, arXiv:2508.16067) describes a model trained on MPtrj and does not report a training-set size or parameter count for this released OAM checkpoint; the 707,569-parameter model size shown here was measured from the loaded checkpoint by FPBench.
§ NEP89 (nep89_20250409) is trained on eleven datasets computed at different quantum-mechanical levels rather than a single reference functional. Its scores on this board therefore measure practical agreement with the MatPES-PBE reference rather than agreement with a single matched reference functional. The eleven datasets and the 537,641-configuration training total are reported in the NEP89 paper.
‡ ALIGNN (alignnff_wt10) is trained on JARVIS-DFT at the OptB88vdW level, not PBE. It is the only potential on this board whose training reference functional differs from the evaluation reference, so its scores measure agreement with the FPBench MatPES-PBE reference rather than a functional-matched fitting error.
| FP | Average errorΔ|F|MAE / RMSE(eV/Å) ↓ | Large-force-error atomsFrac(|Δ|F|| > 1 eV/Å)(%) ↓ | FE atomsΔ|F|MAE / RMSE(eV/Å) ↓ | FE atomsΔθMAE / RMSE(°) ↓ | Highly accurate force predictionsFrac(|Δ|F|| < 0.01 eV/Å)(%) ↑ | Joint force magnitude-angle accuracyFrac(<0.01 eV/Å& <1°/20°)(%) ↑ | Average errorΔθMAE / RMSE(°) ↓ | Excluding large-error atomsΔ|F|MAE / RMSE(eV/Å), on |Δ|F|| < 1 eV/Å ↓ | Version (training) |
|---|---|---|---|---|---|---|---|---|---|
| M3GNet-MatPES-r2SCAN training | 0.23 / 0.65 | 2.87 | 0.41 / 1.07 | 18 / 27 | 5.93 | 0.80 / 3.48 | 30 / 48 | 0.19 / 0.27 | v2025.1 (MatPES-r2SCAN) |
| TensorNet-MatPES-r2SCAN training | 0.19 / 0.71 | 1.64 | 0.33 / 1.17 | 13 / 19 | 6.39 | 0.79 / 4.19 | 26 / 45 | 0.17 / 0.24 | v2025.1 (MatPES-r2SCAN) |
| MACE-MatPES-r2SCAN training | 0.12 / 1.34 | 0.54 | 0.20 / 2.27 | 8 / 12 | 12.82 | 2.33 / 10.35 | 17 / 33 | 0.10 / 0.17 | >=v0.3.10 (MatPES-r2SCAN (fine-tuned)) |
| FP | Average errorΔ|F|MAE / RMSE(eV/Å) ↓ | Large-force-error atomsFrac(|Δ|F|| > 1 eV/Å)(%) ↓ | FE atomsΔ|F|MAE / RMSE(eV/Å) ↓ | FE atomsΔθMAE / RMSE(°) ↓ | Highly accurate force predictionsFrac(|Δ|F|| < 0.01 eV/Å)(%) ↑ | Joint force magnitude-angle accuracyFrac(<0.01 eV/Å& <1°/20°)(%) ↑ | Average errorΔθMAE / RMSE(°) ↓ | Excluding large-error atomsΔ|F|MAE / RMSE(eV/Å), on |Δ|F|| < 1 eV/Å ↓ | Version (training) |
|---|---|---|---|---|---|---|---|---|---|
| MACE OOD | 0.34 / 2.83 | 5.72 | 0.40 / 3.16 | 6 / 9 | 4.08 | 0.27 / 3.71 | 8 / 14 | 0.21 / 0.30 | >=v0.3.10 (MACE-MPA-0, medium) (MPtrj + sAlex) |
| CHGNet OOD | 0.76 / 4.00 | 18.50 | 0.90 / 4.46 | 9 / 15 | 1.81 | 0.03 / 1.34 | 13 / 22 | 0.34 / 0.42 | v0.3.0 (MPtrj) |
| M3GNet OOD | 0.91 / 1.96 | 27.66 | 1.06 / 2.18 | 13 / 21 | 1.30 | 0.01 / 0.81 | 19 / 30 | 0.37 / 0.46 | MP-2021.2.8-PES (MP-2021.2.8) |
| UMA training | 0.09 / 0.22 | 0.51 | 0.10 / 0.24 | 2 / 5 | 13.14 | 4.76 / 12.97 | 4 / 8 | 0.08 / 0.14 | s-1p1 (OC20+ |
| M3GNet-MatPES OOD | 0.55 / 2.18 | 12.43 | 0.64 / 2.43 | 9 / 14 | 2.30 | 0.06 / 1.79 | 13 / 22 | 0.29 / 0.38 | v2025.1 (MatPES-PBE) |
| TensorNet-MatPES OOD | 0.60 / 4.04 | 12.19 | 0.70 / 4.52 | 8 / 12 | 2.76 | 0.11 / 2.30 | 11 / 19 | 0.27 / 0.36 | v2025.1 (MatPES-PBE) |
| MACE-MatPES pretraining | 0.43 / 2.25 | 8.94 | 0.51 / 2.51 | 6 / 9 | 4.12 | 0.31 / 3.74 | 8 / 15 | 0.22 / 0.31 | >=v0.3.10 (OMat24 → MatPES-PBE) |
| Orb training | 0.15 / 0.63 | 1.47 | 0.17 / 0.71 | 3 / 6 | 9.07 | 2.13 / 8.84 | 5 / 10 | 0.12 / 0.19 | orb-v3-conservative-inf-omat-20250404 (OMat24) |
| SevenNet training | 0.16 / 0.37 | 1.67 | 0.19 / 0.41 | 3 / 6 | 7.68 | 1.41 / 7.44 | 5 / 10 | 0.14 / 0.21 | 7net-mf-ompa (modal mpa) (OMat24 + sAlex + MPtrj) |
| MatterSim OOD | 0.26 / 0.89 | 3.17 | 0.30 / 0.98 | 5 / 9 | 4.21 | 0.34 / 3.81 | 8 / 15 | 0.20 / 0.27 | MatterSim-v1.0.0-5M (MatterSim dataset (PBE)) |
| Nequix training | 0.25 / 2.33 | 3.21 | 0.29 / 2.61 | 4 / 7 | 5.88 | 0.69 / 5.61 | 6 / 11 | 0.17 / 0.25 | nequix-oam-1 (OMat24 + sAlex + MPtrj) |
| GPTFF OOD | 0.99 / 3.00 | 27.60 | 1.18 / 3.35 | 12 / 19 | 1.18 | 0.01 / 0.72 | 17 / 27 | 0.39 / 0.47 | gptff_v2 (Atomly (PBE)) |
| NEP89 OOD | 0.70 / 2.71 | 14.29 | 0.83 / 3.02 | 10 / 16 | 2.24 | 0.05 / 1.71 | 14 / 24 | 0.30 / 0.38 | nep89_20250409 (mixed QM levels) |
| DPA4 training | 0.11 / 3.56 | 0.46 | 0.12 / 3.98 | 2 / 5 | 16.64 | 7.50 / 16.51 | 3 / 7 | 0.07 / 0.12 | DPA4-Plus-OMat24-v20260805 (OMat24) |
| GRACE pretraining | 0.17 / 1.67 | 1.98 | 0.20 / 1.87 | 3 / 6 | 10.31 | 2.83 / 10.14 | 4 / 9 | 0.11 / 0.19 | GRACE-3L-OMAT-large-ft-AM (OMat24 → sAlex + MPtrj) |
| eqV2 pretraining | 0.17 / 1.81 | 2.13 | 0.20 / 2.03 | 3 / 6 | 10.10 | 2.66 / 9.94 | 4 / 9 | 0.12 / 0.19 | eqV2_31M_omat_mp_salex (OMat24 → MPtrj + sAlex) |
| eSEN pretraining | 0.17 / 2.51 | 1.97 | 0.20 / 2.80 | 2 / 6 | 11.52 | 3.73 / 11.37 | 4 / 8 | 0.11 / 0.18 | esen_30m_oam (OMat24 → MPtrj + sAlex) |
Evaluated for atoms with |FDFT| > 0.01 eV/Å (except the all-atom average-error analysis, which uses every atom); far-from-equilibrium (FE) atoms are |FDFT| > 1 eV/Å.
| Name | Metrics |
|---|---|
| Average error | Force-magnitude error Δ|F| and force-angle error Δθ, reported as MAE/RMSE over all atoms or a selected subset. |
| Cumulative distribution functions (CDFs) of force errors | CDFs of |Δ|F||, Δθ, and norm of the force-vector error evec, over all atoms or a selected subset. |
| Highly accurate force predictions (small-force-error atoms) | Fraction of atoms with very small values of |Δ|F|| and Δθ below threshold (e.g. |Δ|F|| < 0.01 eV/Å). |
| Joint force magnitude-angle accuracy | Fraction of atoms with simultaneously small values of |Δ|F|| and Δθ (e.g. |Δ|F|| < 0.01 eV/Å and Δθ < 1° or 20°). |
| Force-magnitude error excluding large-error atoms | MAE/RMSE evaluated after excluding atoms with force-magnitude errors > 1 eV/Å, showing model accuracy outside the large-error tail. Reported on the summary leaderboard as Δ|F| MAE/RMSE on |Δ|F|| < 1 eV/Å. |
| Large-force-error atoms | Fraction of atoms with high values of |Δ|F|| and Δθ (e.g. |Δ|F|| > 0.5 eV/Å). |
| Force errors on far-from-equilibrium (FE) atoms | MAE/RMSE evaluated for Δ|F|, Δθ over the FE atoms selected as |FDFT| > 1 eV/Å. Fraction of FE atoms with relative force-magnitude error rF below increasing thresholds. |
These metrics are complementary and should be interpreted together rather than combined into a single overall ranking. Click any column heading to sort by that metric.
Per-dataset tables for the metrics defined above. These follow the dataset tab selected with the leaderboard, currently MatPES-PBE.
| FP | < 0.01 eV/Å | < 0.02 eV/Å | < 0.05 eV/Å | < 0.07 eV/Å | < 0.1 eV/Å | < 0.2 eV/Å | < 0.5 eV/Å |
|---|---|---|---|---|---|---|---|
| Orb | 12.58 | 22.48 | 42.10 | 50.57 | 59.73 | 76.48 | 92.33 |
| SevenNet | 11.22 | 20.50 | 39.55 | 47.96 | 57.15 | 74.22 | 91.05 |
| MatterSim | 7.91 | 15.08 | 31.85 | 40.16 | 49.92 | 69.47 | 89.81 |
| Nequix | 10.74 | 19.60 | 37.71 | 45.75 | 54.64 | 71.77 | 89.99 |
| MACE | 8.34 | 15.92 | 33.11 | 41.21 | 50.31 | 68.03 | 88.05 |
| CHGNet | 5.61 | 10.91 | 23.93 | 30.79 | 39.06 | 57.21 | 81.66 |
| GPTFF | 4.66 | 9.01 | 19.72 | 25.31 | 32.16 | 48.06 | 73.62 |
| M3GNet | 4.56 | 8.88 | 19.54 | 25.30 | 32.42 | 48.81 | 73.45 |
| UMA | 15.13 | 26.40 | 46.86 | 55.13 | 63.72 | 78.73 | 92.22 |
| ALIGNN | 2.55 | 5.04 | 11.66 | 15.56 | 20.87 | 34.85 | 58.48 |
| M3GNet-MatPES | 7.14 | 13.87 | 30.42 | 38.99 | 49.20 | 69.76 | 90.64 |
| TensorNet-MatPES | 7.66 | 14.81 | 32.53 | 41.80 | 52.86 | 74.74 | 93.92 |
| MACE-MatPES | 19.02 | 33.50 | 58.92 | 68.47 | 77.66 | 90.97 | 98.47 |
| NEP89 | 3.20 | 6.24 | 14.64 | 19.80 | 26.96 | 46.32 | 77.00 |
| DPA4 | 19.67 | 32.44 | 53.44 | 61.36 | 69.36 | 82.38 | 93.87 |
| GRACE | 17.77 | 29.16 | 48.80 | 56.67 | 64.92 | 79.31 | 92.65 |
| eqV2 | 14.57 | 26.67 | 48.24 | 56.65 | 65.24 | 79.61 | 92.70 |
| eSEN | 18.63 | 30.91 | 51.50 | 59.39 | 67.40 | 80.79 | 93.04 |
| FP | < 0.01 eV/Å | < 0.02 eV/Å | < 0.05 eV/Å | < 0.07 eV/Å | < 0.1 eV/Å | < 0.2 eV/Å | < 0.5 eV/Å |
|---|---|---|---|---|---|---|---|
| Orb | |||||||
| SevenNet | |||||||
| MatterSim | |||||||
| Nequix | |||||||
| MACE | |||||||
| CHGNet | |||||||
| GPTFF | |||||||
| M3GNet | |||||||
| UMA | |||||||
| ALIGNN | |||||||
| M3GNet-MatPES | |||||||
| TensorNet-MatPES | |||||||
| MACE-MatPES | |||||||
| NEP89 | |||||||
| DPA4 | |||||||
| GRACE | |||||||
| eqV2 | |||||||
| eSEN |
| FP | > 0.5 eV/Å | > 1 eV/Å | > 2 eV/Å | > 3 eV/Å | > 4 eV/Å | > 5 eV/Å | > 7 eV/Å | > 10 eV/Å |
|---|---|---|---|---|---|---|---|---|
| Orb | 7.672 | 2.001 | 0.307 | 0.118 | 0.063 | 0.041 | 0.024 | 0.015 |
| SevenNet | 8.948 | 2.632 | 0.530 | 0.223 | 0.119 | 0.075 | 0.041 | 0.025 |
| MatterSim | 10.191 | 2.653 | 0.377 | 0.142 | 0.085 | 0.059 | 0.037 | 0.024 |
| Nequix | 10.007 | 2.826 | 0.464 | 0.158 | 0.078 | 0.050 | 0.030 | 0.019 |
| MACE | 11.954 | 3.438 | 0.590 | 0.194 | 0.083 | 0.046 | 0.025 | 0.018 |
| CHGNet | 18.338 | 5.707 | 0.945 | 0.290 | 0.122 | 0.069 | 0.036 | 0.023 |
| GPTFF | 26.381 | 9.821 | 2.154 | 0.727 | 0.318 | 0.169 | 0.070 | 0.036 |
| M3GNet | 26.553 | 10.789 | 2.757 | 1.056 | 0.493 | 0.261 | 0.103 | 0.035 |
| UMA | 7.785 | 2.686 | 1.055 | 0.808 | 0.712 | 0.654 | 0.583 | 0.520 |
| ALIGNN | 41.519 | 24.231 | 12.236 | 8.083 | 6.027 | 4.793 | 3.366 | 2.304 |
| M3GNet-MatPES | 9.363 | 2.355 | 0.452 | 0.159 | 0.073 | 0.040 | 0.020 | 0.013 |
| TensorNet-MatPES | 6.080 | 1.196 | 0.175 | 0.065 | 0.043 | 0.033 | 0.024 | 0.018 |
| MACE-MatPES | 1.535 | 0.249 | 0.039 | 0.021 | 0.017 | 0.014 | 0.011 | 0.010 |
| NEP89 | 22.997 | 7.603 | 1.730 | 0.633 | 0.318 | 0.205 | 0.127 | 0.085 |
| DPA4 | 6.130 | 1.636 | 0.258 | 0.119 | 0.081 | 0.064 | 0.046 | 0.034 |
| GRACE | 7.349 | 2.076 | 0.284 | 0.107 | 0.061 | 0.042 | 0.027 | 0.019 |
| eqV2 | 7.301 | 1.981 | 0.219 | 0.076 | 0.042 | 0.032 | 0.022 | 0.018 |
| eSEN | 6.959 | 1.928 | 0.249 | 0.101 | 0.062 | 0.046 | 0.032 | 0.022 |
| FP | All atoms | > 0.01 eV/Å | > 0.05 eV/Å | > 0.1 eV/Å | > 0.2 eV/Å | > 0.5 eV/Å | > 0.7 eV/Å | > 1 eV/Å | > 2 eV/Å |
|---|---|---|---|---|---|---|---|---|---|
| Atoms with |FDFT| > threshold | 100.0% | 97.0% | 86.0% | 76.7% | 66.8% | 48.7% | 38.7% | 26.8% | 8.3% |
| Orb | |||||||||
| SevenNet | |||||||||
| MatterSim | |||||||||
| Nequix | |||||||||
| MACE | |||||||||
| CHGNet | |||||||||
| GPTFF | |||||||||
| M3GNet | |||||||||
| UMA | |||||||||
| ALIGNN | |||||||||
| M3GNet-MatPES | |||||||||
| TensorNet-MatPES | |||||||||
| MACE-MatPES | |||||||||
| NEP89 | |||||||||
| DPA4 | |||||||||
| GRACE | |||||||||
| eqV2 | |||||||||
| eSEN |
| FP | <0.01 | <0.05 | <0.1 | <0.2 | <0.3 | <0.4 | <0.5 | <1 | <2 |
|---|---|---|---|---|---|---|---|---|---|
| Orb | 8.48 | 35.23 | 55.66 | 76.18 | 85.29 | 89.88 | 92.51 | 99.52 | 99.91 |
| SevenNet | 6.36 | 28.70 | 48.39 | 70.65 | 81.61 | 87.34 | 90.60 | 99.20 | 99.80 |
| MatterSim | 5.38 | 24.74 | 43.34 | 66.70 | 79.32 | 86.37 | 90.43 | 99.45 | 99.91 |
| Nequix | 5.00 | 23.15 | 40.94 | 64.35 | 78.00 | 85.78 | 90.11 | 99.51 | 99.91 |
| MACE | 4.00 | 18.90 | 34.56 | 57.18 | 71.82 | 81.26 | 87.22 | 99.29 | 99.89 |
| CHGNet | 1.91 | 9.43 | 18.40 | 35.36 | 50.82 | 64.44 | 75.65 | 99.47 | 99.93 |
| GPTFF | 0.76 | 3.85 | 7.78 | 16.10 | 25.90 | 37.50 | 50.63 | 99.59 | 99.92 |
| M3GNet | 1.07 | 5.35 | 10.54 | 20.97 | 31.65 | 42.92 | 54.67 | 98.34 | 99.63 |
| UMA | 11.07 | 42.04 | 62.00 | 78.93 | 85.63 | 89.11 | 91.25 | 97.98 | 98.59 |
| ALIGNN | 0.58 | 2.84 | 5.67 | 11.57 | 17.92 | 24.98 | 32.99 | 81.91 | 87.76 |
| M3GNet-MatPES | 3.94 | 19.06 | 35.25 | 59.04 | 74.69 | 84.77 | 91.25 | 99.83 | 99.99 |
| TensorNet-MatPES | 5.22 | 24.87 | 45.18 | 71.85 | 85.78 | 92.88 | 96.41 | 99.92 | 100.00 |
| MACE-MatPES | 13.94 | 52.54 | 75.30 | 91.75 | 96.65 | 98.40 | 99.12 | 99.99 | 100.00 |
| NEP89 | 2.12 | 10.44 | 20.58 | 39.30 | 55.17 | 68.11 | 78.23 | 98.08 | 99.69 |
| DPA4 | 15.86 | 52.02 | 70.59 | 83.70 | 88.73 | 91.50 | 93.25 | 99.55 | 99.89 |
| GRACE | 10.69 | 41.74 | 62.53 | 80.06 | 87.01 | 90.51 | 92.59 | 99.64 | 99.94 |
| eqV2 | 11.71 | 45.26 | 65.84 | 81.73 | 87.67 | 90.70 | 92.56 | 99.70 | 99.97 |
| eSEN | 13.37 | 48.16 | 67.73 | 82.37 | 87.88 | 90.76 | 92.55 | 99.63 | 99.94 |
| FP | < 0.01 eV/Å | < 0.02 eV/Å | < 0.05 eV/Å | < 0.07 eV/Å | < 0.1 eV/Å | < 0.2 eV/Å | < 0.5 eV/Å |
|---|---|---|---|---|---|---|---|
| M3GNet-MatPES-r2SCAN | 5.93 | 11.61 | 26.34 | 34.42 | 44.42 | 65.71 | 88.81 |
| TensorNet-MatPES-r2SCAN | 6.39 | 12.45 | 28.17 | 36.78 | 47.53 | 70.16 | 92.17 |
| MACE-MatPES-r2SCAN | 12.82 | 23.81 | 46.83 | 56.78 | 67.22 | 84.61 | 96.91 |
| FP | < 0.01 eV/Å | < 0.02 eV/Å | < 0.05 eV/Å | < 0.07 eV/Å | < 0.1 eV/Å | < 0.2 eV/Å | < 0.5 eV/Å |
|---|---|---|---|---|---|---|---|
| M3GNet-MatPES-r2SCAN | |||||||
| TensorNet-MatPES-r2SCAN | |||||||
| MACE-MatPES-r2SCAN |
| FP | > 0.5 eV/Å | > 1 eV/Å | > 2 eV/Å | > 3 eV/Å | > 4 eV/Å | > 5 eV/Å | > 7 eV/Å | > 10 eV/Å |
|---|---|---|---|---|---|---|---|---|
| M3GNet-MatPES-r2SCAN | 11.189 | 2.866 | 0.538 | 0.189 | 0.085 | 0.043 | 0.018 | 0.009 |
| TensorNet-MatPES-r2SCAN | 7.834 | 1.636 | 0.254 | 0.097 | 0.058 | 0.041 | 0.024 | 0.014 |
| MACE-MatPES-r2SCAN | 3.091 | 0.543 | 0.087 | 0.042 | 0.028 | 0.021 | 0.012 | 0.007 |
| FP | All atoms | > 0.01 eV/Å | > 0.05 eV/Å | > 0.1 eV/Å | > 0.2 eV/Å | > 0.5 eV/Å | > 0.7 eV/Å | > 1 eV/Å | > 2 eV/Å |
|---|---|---|---|---|---|---|---|---|---|
| Atoms with |FDFT| > threshold | 100.0% | 96.4% | 89.0% | 84.1% | 76.6% | 57.8% | 46.6% | 33.3% | 10.8% |
| M3GNet-MatPES-r2SCAN | |||||||||
| TensorNet-MatPES-r2SCAN | |||||||||
| MACE-MatPES-r2SCAN |
| FP | <0.01 | <0.05 | <0.1 | <0.2 | <0.3 | <0.4 | <0.5 | <1 | <2 |
|---|---|---|---|---|---|---|---|---|---|
| M3GNet-MatPES-r2SCAN | 3.98 | 19.20 | 35.50 | 59.79 | 75.39 | 85.42 | 91.72 | 99.83 | 99.99 |
| TensorNet-MatPES-r2SCAN | 4.99 | 23.98 | 43.96 | 70.70 | 85.07 | 92.48 | 96.22 | 99.89 | 99.99 |
| MACE-MatPES-r2SCAN | 10.16 | 42.42 | 65.96 | 86.92 | 94.47 | 97.44 | 98.72 | 99.97 | 100.00 |
| FP | < 0.01 eV/Å | < 0.02 eV/Å | < 0.05 eV/Å | < 0.07 eV/Å | < 0.1 eV/Å | < 0.2 eV/Å | < 0.5 eV/Å |
|---|---|---|---|---|---|---|---|
| MACE | 4.08 | 8.15 | 19.75 | 26.77 | 36.09 | 58.16 | 83.88 |
| CHGNet | 1.81 | 3.60 | 8.89 | 12.32 | 17.23 | 31.73 | 60.55 |
| M3GNet | 1.30 | 2.60 | 6.49 | 9.05 | 12.85 | 24.49 | 50.02 |
| UMA | 13.14 | 24.96 | 51.43 | 63.05 | 74.57 | 90.29 | 98.05 |
| M3GNet-MatPES | 2.30 | 4.59 | 11.30 | 15.63 | 21.87 | 39.76 | 70.26 |
| TensorNet-MatPES | 2.76 | 5.52 | 13.57 | 18.68 | 25.78 | 44.65 | 72.94 |
| MACE-MatPES | 4.12 | 8.19 | 19.62 | 26.34 | 35.07 | 55.22 | 79.65 |
| Orb | 9.07 | 17.65 | 38.85 | 49.42 | 61.28 | 81.47 | 95.12 |
| SevenNet | 7.68 | 15.08 | 34.19 | 44.29 | 56.09 | 77.56 | 93.83 |
| MatterSim | 4.21 | 8.36 | 20.40 | 27.90 | 38.00 | 62.41 | 89.04 |
| Nequix | 5.88 | 11.64 | 27.25 | 36.03 | 46.91 | 69.29 | 90.02 |
| GPTFF | 1.18 | 2.33 | 5.85 | 8.15 | 11.61 | 22.64 | 48.84 |
| NEP89 | 2.24 | 4.47 | 11.07 | 15.33 | 21.43 | 38.86 | 68.60 |
| DPA4 | 16.64 | 31.00 | 60.23 | 71.54 | 81.69 | 93.44 | 98.47 |
| GRACE | 10.31 | 19.91 | 42.92 | 53.87 | 65.38 | 82.81 | 94.40 |
| eqV2 | 10.10 | 19.47 | 41.92 | 52.66 | 64.24 | 82.00 | 94.03 |
| eSEN | 11.52 | 22.01 | 45.86 | 56.70 | 67.77 | 83.78 | 94.52 |
| FP | < 0.01 eV/Å | < 0.02 eV/Å | < 0.05 eV/Å | < 0.07 eV/Å | < 0.1 eV/Å | < 0.2 eV/Å | < 0.5 eV/Å |
|---|---|---|---|---|---|---|---|
| MACE | |||||||
| CHGNet | |||||||
| M3GNet | |||||||
| UMA | |||||||
| M3GNet-MatPES | |||||||
| TensorNet-MatPES | |||||||
| MACE-MatPES | |||||||
| Orb | |||||||
| SevenNet | |||||||
| MatterSim | |||||||
| Nequix | |||||||
| GPTFF | |||||||
| NEP89 | |||||||
| DPA4 | |||||||
| GRACE | |||||||
| eqV2 | |||||||
| eSEN |
| FP | > 0.5 eV/Å | > 1 eV/Å | > 2 eV/Å | > 3 eV/Å | > 4 eV/Å | > 5 eV/Å | > 7 eV/Å | > 10 eV/Å |
|---|---|---|---|---|---|---|---|---|
| MACE | 16.115 | 5.717 | 1.595 | 0.716 | 0.407 | 0.266 | 0.145 | 0.082 |
| CHGNet | 39.447 | 18.502 | 6.821 | 3.545 | 2.170 | 1.461 | 0.775 | 0.376 |
| M3GNet | 49.977 | 27.656 | 10.847 | 5.111 | 2.736 | 1.665 | 0.729 | 0.279 |
| UMA | 1.947 | 0.514 | 0.121 | 0.046 | 0.024 | 0.014 | 0.007 | 0.003 |
| M3GNet-MatPES | 29.737 | 12.432 | 4.042 | 1.927 | 1.117 | 0.719 | 0.374 | 0.191 |
| TensorNet-MatPES | 27.063 | 12.192 | 4.689 | 2.561 | 1.615 | 1.114 | 0.631 | 0.337 |
| MACE-MatPES | 20.351 | 8.939 | 3.191 | 1.582 | 0.938 | 0.620 | 0.328 | 0.172 |
| Orb | 4.879 | 1.472 | 0.415 | 0.198 | 0.122 | 0.083 | 0.041 | 0.021 |
| SevenNet | 6.168 | 1.667 | 0.346 | 0.124 | 0.065 | 0.039 | 0.020 | 0.010 |
| MatterSim | 10.965 | 3.171 | 0.876 | 0.423 | 0.262 | 0.180 | 0.106 | 0.062 |
| Nequix | 9.981 | 3.213 | 0.880 | 0.418 | 0.256 | 0.178 | 0.102 | 0.058 |
| GPTFF | 51.158 | 27.599 | 11.181 | 5.839 | 3.482 | 2.252 | 1.098 | 0.475 |
| NEP89 | 31.403 | 14.290 | 5.879 | 3.558 | 2.489 | 1.868 | 1.168 | 0.664 |
| DPA4 | 1.525 | 0.456 | 0.130 | 0.064 | 0.045 | 0.035 | 0.027 | 0.021 |
| GRACE | 5.596 | 1.985 | 0.640 | 0.327 | 0.211 | 0.148 | 0.087 | 0.053 |
| eqV2 | 5.973 | 2.131 | 0.674 | 0.343 | 0.219 | 0.157 | 0.092 | 0.054 |
| eSEN | 5.483 | 1.974 | 0.631 | 0.321 | 0.204 | 0.143 | 0.084 | 0.049 |
| FP | All atoms | > 0.01 eV/Å | > 0.05 eV/Å | > 0.1 eV/Å | > 0.2 eV/Å | > 0.5 eV/Å | > 0.7 eV/Å | > 1 eV/Å | > 2 eV/Å |
|---|---|---|---|---|---|---|---|---|---|
| Atoms with |FDFT| > threshold | 100.0% | 100.0% | 100.0% | 99.8% | 99.0% | 93.4% | 88.2% | 80.0% | 56.6% |
| MACE | |||||||||
| CHGNet | |||||||||
| M3GNet | |||||||||
| UMA | |||||||||
| M3GNet-MatPES | |||||||||
| TensorNet-MatPES | |||||||||
| MACE-MatPES | |||||||||
| Orb | |||||||||
| SevenNet | |||||||||
| MatterSim | |||||||||
| Nequix | |||||||||
| GPTFF | |||||||||
| NEP89 | |||||||||
| DPA4 | |||||||||
| GRACE | |||||||||
| eqV2 | |||||||||
| eSEN |
| FP | <0.01 | <0.05 | <0.1 | <0.2 | <0.3 | <0.4 | <0.5 | <1 | <2 |
|---|---|---|---|---|---|---|---|---|---|
| MACE | 9.79 | 44.27 | 70.76 | 90.48 | 96.12 | 98.14 | 99.03 | 99.93 | 100.00 |
| CHGNet | 3.81 | 18.72 | 36.13 | 63.60 | 80.63 | 89.88 | 94.69 | 99.65 | 99.98 |
| M3GNet | 3.04 | 14.97 | 29.03 | 52.11 | 68.17 | 78.78 | 85.60 | 97.18 | 99.70 |
| UMA | 36.87 | 83.78 | 94.47 | 98.38 | 99.22 | 99.57 | 99.74 | 99.97 | 100.00 |
| M3GNet-MatPES | 5.65 | 27.05 | 49.01 | 75.24 | 87.45 | 93.57 | 96.77 | 99.86 | 99.99 |
| TensorNet-MatPES | 6.26 | 29.94 | 53.08 | 78.36 | 89.23 | 94.42 | 97.12 | 99.87 | 99.98 |
| MACE-MatPES | 9.02 | 40.80 | 65.39 | 85.61 | 92.94 | 96.43 | 98.24 | 99.97 | 100.00 |
| Orb | 24.22 | 73.89 | 90.27 | 97.34 | 98.84 | 99.39 | 99.65 | 99.97 | 100.00 |
| SevenNet | 19.60 | 69.30 | 88.57 | 96.94 | 98.73 | 99.36 | 99.63 | 99.97 | 100.00 |
| MatterSim | 12.48 | 51.41 | 75.89 | 92.48 | 97.11 | 98.69 | 99.33 | 99.95 | 100.00 |
| Nequix | 13.74 | 57.70 | 82.58 | 95.30 | 98.15 | 99.11 | 99.52 | 99.97 | 100.00 |
| GPTFF | 2.35 | 11.85 | 23.74 | 46.38 | 65.23 | 78.45 | 86.98 | 99.76 | 99.97 |
| NEP89 | 5.36 | 25.51 | 46.00 | 71.78 | 84.89 | 91.71 | 95.43 | 99.77 | 99.98 |
| DPA4 | 44.68 | 88.26 | 95.99 | 98.64 | 99.31 | 99.60 | 99.76 | 99.98 | 100.00 |
| GRACE | 25.59 | 76.04 | 91.71 | 97.53 | 98.83 | 99.35 | 99.61 | 99.97 | 100.00 |
| eqV2 | 23.96 | 75.40 | 91.59 | 97.48 | 98.78 | 99.32 | 99.59 | 99.97 | 100.00 |
| eSEN | 27.77 | 78.07 | 92.51 | 97.64 | 98.84 | 99.35 | 99.61 | 99.97 | 100.00 |
Model versions and official sources for the evaluated FPs are documented once on the FPBench home page.
Paired Cartesian DFT and FP forces
or
Full generator calculations on a dataset
↓
standardized force_results
↓
validation and analysis
↓
FPBench force-error tablesBuild standardized force results directly from paired Cartesian DFT and FP forces. Call build_force_results(dft_forces, fp_forces, structure_ids=None) from scripts/force_results.py with your own data.
Use a generator notebook as a template for full dataset/cluster generation. Register your FP's checkpoint and calculator setup in one of the three generator notebooks' POTENTIAL_REGISTRY, run the jobs on your cluster (the notebooks never submit jobs by themselves), then run the matching analysis notebook -- your FP appears in every table above.
git clone https://github.com/mogroupumd/FPBench.git
cd FPBench/Force_error
pip install -r requirements.txt
pip install jupyterlab
jupyter lab analysis/force_error_analysis_matpes_pbe.ipynbSee the Force_error README for the full quick start, the generator and analysis notebooks for complete operational detail, and the included Cartesian-force example for a small runnable slice of real data.
Interested in evaluating a new foundation potential, or having it considered for inclusion in FPBench? See our Adding a Potential guide to integrate and evaluate a new model with FPBench. For inclusion in the public leaderboard, please contact Prof. Yifei Mo at yfmo@umd.edu with the model name, version/checkpoint, and a link to the official implementation or model weights.