▼ lower is better ▲ higher is better green = best in column cell shading: green = better, gold = middle, red = worse identifiers, badges, and thresholds are never colored

The summary leaderboard highlights average force error together with application-oriented metrics for highly accurate force predictions, large-force-error atoms, and far-from-equilibrium (FE) atoms. Highly accurate force predictions have force-magnitude errors |Δ|F|| < 0.01 eV/Å, large-force-error atoms have |Δ|F|| > 1 eV/Å, and FE atoms are defined by |F_DFT| > 1 eV/Å. Full definitions are in Error-decomposition metrics, below the leaderboard.

The per-column best value (green) updates when you click a column header. FP badges show whether that FP's own training data included this dataset's functional/source (training), included it only during pretraining before the released checkpoint was fine-tuned onto a different dataset (pretraining), did not include it (OOD), or is a fine-tune of an OOD base model onto this dataset (fine-tuned).

Leaderboard

Combined summary
FP	Average errorΔ\|F\|MAE / RMSE(eV/Å) ↓	Large-force-error atomsFrac(\|Δ\|F\|\| > 1 eV/Å)(%) ↓	FE atomsΔ\|F\|MAE / RMSE(eV/Å) ↓	FE atomsΔθMAE / RMSE(°) ↓	Highly accurate force predictionsFrac(\|Δ\|F\|\| < 0.01 eV/Å)(%) ↑	Joint force magnitude-angle accuracyFrac(<0.01 eV/Å& <1°/20°)(%) ↑	Average errorΔθMAE / RMSE(°) ↓	Excluding large-error atomsΔ\|F\|MAE / RMSE(eV/Å), on \|Δ\|F\|\| < 1 eV/Å ↓	Version (training)
Orb OOD	0.17 / 1.12	2.00	0.31 / 2.09	12 / 24	12.6	1.4 / 8.9	30 / 52	0.14 / 0.22	orb-v3-conservative-inf-omat-20250404 (OMat24)
SevenNet OOD	0.19 / 0.91	2.63	0.39 / 1.70	15 / 28	11.2	1.6 / 7.4	33 / 55	0.15 / 0.24	7net-mf-ompa (modal mpa) (OMat24 + sAlex + MPTrj)
SevenNet-0 OOD	0.30 / 2.23	5.14	0.68 / 4.21	24 / 38	7.4	0.99 / 3.5	42 / 63	0.20 / 0.30	7net-0 (SevenNet_0__11Jul2024) (MPTrj)
SevenNet-Nano training	0.24 / 1.17	3.31	0.47 / 2.18	18 / 28	6.0	0.66 / 3.3	40 / 62	0.19 / 0.28	7net-nano-6.0 (7net-Omni distillation set)
MatterSim OOD	0.21 / 1.29	2.65	0.40 / 2.41	16 / 28	7.9	1.00 / 4.5	37 / 59	0.17 / 0.26	MatterSim-v1.0.0-5M (MatterSim dataset (PBE))
MatterSim-1M OOD	0.23 / 1.05	2.85	0.41 / 1.95	17 / 28	6.6	0.74 / 3.6	39 / 61	0.18 / 0.27	MatterSim-v1.0.0-1M (MatterSim dataset (PBE))
Nequix OOD	0.21 / 1.52	2.83	0.43 / 2.86	15 / 27	10.7	1.6 / 6.9	33 / 55	0.16 / 0.25	nequix-oam-1 (OMat24 + sAlex + MPTrj)
MACE OOD	0.23 / 1.93	3.44	0.49 / 3.65	18 / 29	8.34	1.07 / 4.51	37 / 58	0.17 / 0.27	>=v0.3.10 (MACE-MPA-0, medium) (MPTrj + sAlex)
CHGNet OOD	0.31 / 2.19	5.71	0.70 / 4.14	24 / 36	5.61	0.66 / 2.16	45 / 66	0.22 / 0.32	v0.3.0 (MPTrj)
GPTFF OOD	0.42 / 1.89	9.82	1.00 / 3.55	30 / 43	4.7	0.59 / 1.4	50 / 70	0.26 / 0.36	gptff_v2 (Atomly (PBE))
M3GNet OOD	0.44 / 1.49	10.79	0.98 / 2.76	40 / 56	4.56	0.54 / 1.45	56 / 76	0.26 / 0.36	MP-2021.2.8-PES (MP-2021.2.8)
UMA OOD	0.56 / 7.93	2.69	1.13 / 11.89	12 / 25	15.13	2.79 / 11.44	28 / 51	0.12 / 0.21	s-1p1 (OC20+ODAC23+OMat24+OMC25+OMol25)
ALIGNN OOD functional mismatch	2.08 / 24.80	24.23	5.51 / 46.95	61 / 77	2.5	0.20 / 0.55	70 / 87	0.31 / 0.40	alignnff_wt10 (JARVIS-DFT)OptB88vdW → PBE
M3GNet-MatPES training	0.21 / 1.24	2.35	0.43 / 2.34	18 / 27	7.14	0.92 / 3.75	34 / 53	0.17 / 0.25	v2025.1 (MatPES-PBE)
TensorNet-MatPES training	0.17 / 1.39	1.20	0.33 / 2.63	12 / 18	7.66	0.98 / 4.65	29 / 49	0.15 / 0.22	v2025.1 (MatPES-PBE)
MACE-MatPES training	0.08 / 1.10	0.25	0.16 / 2.09	6 / 10	19.02	3.88 / 15.35	16 / 32	0.07 / 0.13	>=v0.3.10 (MatPES-PBE (fine-tuned))
NEP89 OOD	0.40 / 3.70	7.60	0.74 / 6.99	31 / 44	3.2	0.31 / 1.3	52 / 73	0.27 / 0.36	nep89_20250409 (mixed QM levels)
DPA4 OOD	0.15 / 2.16	1.64	0.27 / 3.71	10 / 22	19.7	4.7 / 16	25 / 48	0.11 / 0.20	DPA4-Plus-OMat24-v20260805 (OMat24)
DPA4-Air OOD	0.15 / 2.17	1.65	0.28 / 4.08	10 / 23	17.2	3.6 / 14	26 / 49	0.11 / 0.20	DPA4-Air-OMat24-v20260805 (OMat24)
GRACE OOD	0.16 / 1.49	2.08	0.30 / 2.81	12 / 25	17.8	3.7 / 14	27 / 50	0.12 / 0.21	GRACE-3L-OMAT-large-ft-AM (OMat24 → sAlex + MPTrj)
GRACE-1L OOD	0.25 / 1.97	3.85	0.54 / 3.73	19 / 31	8.5	1.3 / 4.8	37 / 58	0.18 / 0.27	GRACE-1L-OAM (OMat24 → sAlex + MPTrj)
GRACE-2L OOD	0.24 / 2.58	3.06	0.54 / 4.88	17 / 30	11.4	1.8 / 7.4	33 / 54	0.15 / 0.24	GRACE-2L-OAM (OMat24 → sAlex + MPTrj)
GRACE-2L-large OOD	0.19 / 1.77	2.33	0.37 / 3.35	13 / 26	11.9	1.9 / 8.2	32 / 55	0.14 / 0.23	GRACE-2L-OMAT-large-ft-AM (OMat24 → sAlex + MPTrj)
eqV2 OOD	0.16 / 1.27	1.98	0.28 / 2.39	11 / 25	14.6	2.5 / 12	28 / 51	0.12 / 0.21	eqV2_31M_omat_mp_salex (OMat24 → MPTrj + sAlex)
eSEN OOD	0.15 / 1.38	1.93	0.28 / 2.60	11 / 25	18.6	4.0 / 15	27 / 51	0.12 / 0.21	esen_30m_oam (OMat24 → MPTrj + sAlex)

Notes

Hover or click the model markers (* † ‡ § ¶) in the table for model-specific details.

^* MatterSim’s training dataset is not publicly documented in detail. The model card states only that the released MatterSim-v1.0.0-5M checkpoint was trained on a PBE-based dataset of approximately 6M structures; the dataset is neither named nor released. The smaller MatterSim-v1.0.0-1M checkpoint (MatterSim-1M) is listed on the same model card with a training data size of 3M, and the same caveat applies to it. Its OOD label therefore reflects the absence of any documented MatPES or OMat24 training exposure, not a verified dataset composition.

^† The FP-DeErr checkpoint is nequix-oam-1. The official nequix repository documents this checkpoint as trained on OMat24, sAlex and MPTrj at the DFT (PBE+U) level. The Nequix paper (Koker, Kotak & Smidt, arXiv:2508.16067) describes a model trained on MPTrj and does not report a training-set size or parameter count for this released OAM checkpoint; the 707,569-parameter model size shown here was measured from the loaded checkpoint by FP-DeErr.

^§ NEP89 (nep89_20250409) is trained on eleven datasets computed at different quantum-mechanical levels rather than a single reference functional. Its scores on this board therefore measure practical agreement with the MatPES-PBE reference rather than agreement with a single matched reference functional. The eleven datasets and the 537,641-configuration training total are reported in the NEP89 paper.

^‡ ALIGNN (alignnff_wt10) is trained on JARVIS-DFT at the OptB88vdW level, not PBE. It is the only potential on this board trained entirely at a single functional other than the evaluation reference, so its scores measure agreement with the FP-DeErr MatPES-PBE reference rather than a functional-matched fitting error.

^¶ SevenNet-Nano (7net-nano-6.0) is a 105k-parameter model distilled from 7net-Omni: structures from 7net-Omni’s training databases, including MatPES, MatPES-r2SCAN, OMat24, MPTrj and Alexandria, were relabelled with 7net-Omni’s mpa (PBE) task and used as its training set (arXiv:2604.10887). MatPES and OMat24 structures are therefore inside its training data, so it carries the training tag on both boards. The size of the distillation set is not reported.

Leaderboard

Combined summary
FP	Average errorΔ\|F\|MAE / RMSE(eV/Å) ↓	Large-force-error atomsFrac(\|Δ\|F\|\| > 1 eV/Å)(%) ↓	FE atomsΔ\|F\|MAE / RMSE(eV/Å) ↓	FE atomsΔθMAE / RMSE(°) ↓	Highly accurate force predictionsFrac(\|Δ\|F\|\| < 0.01 eV/Å)(%) ↑	Joint force magnitude-angle accuracyFrac(<0.01 eV/Å& <1°/20°)(%) ↑	Average errorΔθMAE / RMSE(°) ↓	Excluding large-error atomsΔ\|F\|MAE / RMSE(eV/Å), on \|Δ\|F\|\| < 1 eV/Å ↓	Version (training)
M3GNet-MatPES-r2SCAN training	0.23 / 0.65	2.87	0.41 / 1.07	18 / 27	5.93	0.80 / 3.48	30 / 48	0.19 / 0.27	v2025.1 (MatPES-r2SCAN)
TensorNet-MatPES-r2SCAN training	0.19 / 0.71	1.64	0.33 / 1.17	13 / 19	6.39	0.79 / 4.19	26 / 45	0.17 / 0.24	v2025.1 (MatPES-r2SCAN)
MACE-MatPES-r2SCAN training	0.12 / 1.34	0.54	0.20 / 2.27	8 / 12	12.82	2.33 / 10.35	17 / 33	0.10 / 0.17	>=v0.3.10 (MatPES-r2SCAN (fine-tuned))

Leaderboard

Combined summary
FP	Average errorΔ\|F\|MAE / RMSE(eV/Å) ↓	Large-force-error atomsFrac(\|Δ\|F\|\| > 1 eV/Å)(%) ↓	FE atomsΔ\|F\|MAE / RMSE(eV/Å) ↓	FE atomsΔθMAE / RMSE(°) ↓	Highly accurate force predictionsFrac(\|Δ\|F\|\| < 0.01 eV/Å)(%) ↑	Joint force magnitude-angle accuracyFrac(<0.01 eV/Å& <1°/20°)(%) ↑	Average errorΔθMAE / RMSE(°) ↓	Excluding large-error atomsΔ\|F\|MAE / RMSE(eV/Å), on \|Δ\|F\|\| < 1 eV/Å ↓	Version (training)
MACE OOD	0.34 / 2.83	5.72	0.40 / 3.16	6 / 9	4.08	0.27 / 3.71	8 / 14	0.21 / 0.30	>=v0.3.10 (MACE-MPA-0, medium) (MPTrj + sAlex)
CHGNet OOD	0.76 / 4.00	18.50	0.90 / 4.46	9 / 15	1.81	0.03 / 1.34	13 / 22	0.34 / 0.42	v0.3.0 (MPTrj)
M3GNet OOD	0.91 / 1.96	27.66	1.06 / 2.18	13 / 21	1.30	0.01 / 0.81	19 / 30	0.37 / 0.46	MP-2021.2.8-PES (MP-2021.2.8)
UMA training	0.09 / 0.22	0.51	0.10 / 0.24	2 / 5	13.14	4.76 / 12.97	4 / 8	0.08 / 0.14	s-1p1 (OC20+ODAC23+OMat24+OMC25+OMol25)
M3GNet-MatPES OOD	0.55 / 2.18	12.43	0.64 / 2.43	9 / 14	2.30	0.06 / 1.79	13 / 22	0.29 / 0.38	v2025.1 (MatPES-PBE)
TensorNet-MatPES OOD	0.60 / 4.04	12.19	0.70 / 4.52	8 / 12	2.76	0.11 / 2.30	11 / 19	0.27 / 0.36	v2025.1 (MatPES-PBE)
MACE-MatPES pretraining	0.43 / 2.25	8.94	0.51 / 2.51	6 / 9	4.12	0.31 / 3.74	8 / 15	0.22 / 0.31	>=v0.3.10 (OMat24 → MatPES-PBE)
Orb training	0.15 / 0.63	1.47	0.17 / 0.71	3 / 6	9.07	2.13 / 8.84	5 / 10	0.12 / 0.19	orb-v3-conservative-inf-omat-20250404 (OMat24)
SevenNet training	0.16 / 0.37	1.67	0.19 / 0.41	3 / 6	7.68	1.41 / 7.44	5 / 10	0.14 / 0.21	7net-mf-ompa (modal mpa) (OMat24 + sAlex + MPTrj)
SevenNet-0 OOD	0.82 / 6.97	18.94	0.97 / 7.78	10 / 16	2.03	0.04 / 1.56	14 / 22	0.31 / 0.40	7net-0 (SevenNet_0__11Jul2024) (MPTrj)
SevenNet-Nano training	0.26 / 0.82	3.66	0.30 / 0.92	5 / 9	4.61	0.40 / 4.24	8 / 14	0.20 / 0.28	7net-nano-6.0 (7net-Omni distillation set)
MatterSim OOD	0.26 / 0.89	3.17	0.30 / 0.98	5 / 9	4.21	0.34 / 3.81	8 / 15	0.20 / 0.27	MatterSim-v1.0.0-5M (MatterSim dataset (PBE))
MatterSim-1M OOD	0.28 / 1.11	3.69	0.32 / 1.23	6 / 9	3.92	0.28 / 3.48	9 / 16	0.21 / 0.28	MatterSim-v1.0.0-1M (MatterSim dataset (PBE))
Nequix training	0.25 / 2.33	3.21	0.29 / 2.61	4 / 7	5.88	0.69 / 5.61	6 / 11	0.17 / 0.25	nequix-oam-1 (OMat24 + sAlex + MPTrj)
GPTFF OOD	0.99 / 3.00	27.60	1.18 / 3.35	12 / 19	1.18	0.01 / 0.72	17 / 27	0.39 / 0.47	gptff_v2 (Atomly (PBE))
NEP89 OOD	0.70 / 2.71	14.29	0.83 / 3.02	10 / 16	2.24	0.05 / 1.71	14 / 24	0.30 / 0.38	nep89_20250409 (mixed QM levels)
DPA4 training	0.11 / 3.56	0.46	0.12 / 3.98	2 / 5	16.64	7.50 / 16.51	3 / 7	0.07 / 0.12	DPA4-Plus-OMat24-v20260805 (OMat24)
DPA4-Air training	0.11 / 3.63	0.48	0.13 / 4.05	2 / 5	14.63	5.88 / 14.48	3 / 8	0.07 / 0.13	DPA4-Air-OMat24-v20260805 (OMat24)
GRACE pretraining	0.17 / 1.67	1.98	0.20 / 1.87	3 / 6	10.31	2.83 / 10.14	4 / 9	0.11 / 0.19	GRACE-3L-OMAT-large-ft-AM (OMat24 → sAlex + MPTrj)
GRACE-1L pretraining	0.31 / 3.64	4.33	0.36 / 4.07	5 / 9	4.77	0.43 / 4.42	7 / 13	0.20 / 0.28	GRACE-1L-OAM (OMat24 → sAlex + MPTrj)
GRACE-2L pretraining	0.25 / 3.18	3.14	0.30 / 3.56	4 / 7	5.95	0.71 / 5.67	6 / 11	0.17 / 0.25	GRACE-2L-OAM (OMat24 → sAlex + MPTrj)
GRACE-2L-large pretraining	0.17 / 2.11	1.78	0.19 / 2.36	3 / 6	9.20	2.18 / 8.99	4 / 9	0.12 / 0.19	GRACE-2L-OMAT-large-ft-AM (OMat24 → sAlex + MPTrj)
eqV2 pretraining	0.17 / 1.81	2.13	0.20 / 2.03	3 / 6	10.10	2.66 / 9.94	4 / 9	0.12 / 0.19	eqV2_31M_omat_mp_salex (OMat24 → MPTrj + sAlex)
eSEN pretraining	0.17 / 2.51	1.97	0.20 / 2.80	2 / 6	11.52	3.73 / 11.37	4 / 8	0.11 / 0.18	esen_30m_oam (OMat24 → MPTrj + sAlex)

Error-decomposition metrics

Evaluated for atoms with |F_DFT| > 0.01 eV/Å (except the all-atom average-error analysis, which uses every atom); far-from-equilibrium (FE) atoms are |F_DFT| > 1 eV/Å.

Name	Metrics
Average error	Force-magnitude error Δ\|F\| and force-angle error Δθ, reported as MAE/RMSE over all atoms or a selected subset.
Cumulative distribution functions (CDFs) of force errors	CDFs of \|Δ\|F\|\|, Δθ, and norm of the force-vector error e_vec, over all atoms or a selected subset.
Highly accurate force predictions (small-force-error atoms)	Fraction of atoms with very small values of \|Δ\|F\|\| and Δθ below threshold (e.g. \|Δ\|F\|\| < 0.01 eV/Å).
Joint force magnitude-angle accuracy	Fraction of atoms with simultaneously small values of \|Δ\|F\|\| and Δθ (e.g. \|Δ\|F\|\| < 0.01 eV/Å and Δθ < 1° or 20°).
Force-magnitude error excluding large-error atoms	MAE/RMSE evaluated after excluding atoms with force-magnitude errors > 1 eV/Å, showing model accuracy outside the large-error tail. Reported on the summary leaderboard as Δ\|F\| MAE/RMSE on \|Δ\|F\|\| < 1 eV/Å.
Large-force-error atoms	Fraction of atoms with high values of \|Δ\|F\|\| and Δθ (e.g. \|Δ\|F\|\| > 0.5 eV/Å).
Force errors on far-from-equilibrium (FE) atoms	MAE/RMSE evaluated for Δ\|F\|, Δθ over the FE atoms selected as \|F_DFT\| > 1 eV/Å. Fraction of FE atoms with relative force-magnitude error r_F below increasing thresholds.

Notation: Δ|F| = |F_FP| − |F_DFT| (force-magnitude error); |Δ|F|| for CDFs, thresholds, and fraction metrics; Δθ, the angle between the FP and DFT force vectors; e_vec = ||F_FP − F_DFT|| (force-vector error); r_F = |Δ|F|| / |F_DFT| (relative force-magnitude error, FE atoms only). Atoms with zero FP or DFT force are excluded from every metric, since the force angle is undefined in that case.

These metrics are complementary and should be interpreted together rather than combined into a single overall ranking. Click any column heading to sort by that metric.

Detailed results

Per-dataset tables for the metrics defined above. These follow the dataset tab selected with the leaderboard, currently MatPES-PBE.

Highly accurate force predictions

Percentage of evaluated atoms satisfying |Δ|F|| < threshold. Higher is better.

FP	< 0.01 eV/Å	< 0.02 eV/Å	< 0.05 eV/Å	< 0.07 eV/Å	< 0.1 eV/Å	< 0.2 eV/Å	< 0.5 eV/Å
Orb	12.58	22.48	42.10	50.57	59.73	76.48	92.33
SevenNet	11.22	20.50	39.55	47.96	57.15	74.22	91.05
SevenNet-0	7.35	13.88	28.56	35.68	43.93	61.44	83.99
SevenNet-Nano	6.03	11.69	26.06	33.83	43.37	64.07	87.46
MatterSim	7.91	15.08	31.85	40.16	49.92	69.47	89.81
MatterSim-1M	6.60	12.76	28.07	36.16	45.97	66.62	88.87
Nequix	10.74	19.60	37.71	45.75	54.64	71.77	89.99
MACE	8.34	15.92	33.11	41.21	50.31	68.03	88.05
CHGNet	5.61	10.91	23.93	30.79	39.06	57.21	81.66
GPTFF	4.66	9.01	19.72	25.31	32.16	48.06	73.62
M3GNet	4.56	8.88	19.54	25.30	32.42	48.81	73.45
UMA	15.13	26.40	46.86	55.13	63.72	78.73	92.22
ALIGNN	2.55	5.04	11.66	15.56	20.87	34.85	58.48
M3GNet-MatPES	7.14	13.87	30.42	38.99	49.20	69.76	90.64
TensorNet-MatPES	7.66	14.81	32.53	41.80	52.86	74.74	93.92
MACE-MatPES	19.02	33.50	58.92	68.47	77.66	90.97	98.47
NEP89	3.20	6.24	14.64	19.80	26.96	46.32	77.00
DPA4	19.67	32.44	53.44	61.36	69.36	82.38	93.87
DPA4-Air	17.24	29.25	50.32	58.56	67.03	81.29	93.69
GRACE	17.77	29.16	48.80	56.67	64.92	79.31	92.65
GRACE-1L	8.55	16.00	32.57	40.38	49.26	67.14	87.48
GRACE-2L	11.40	20.59	38.77	46.67	55.34	72.13	90.02
GRACE-2L-large	11.86	21.52	41.24	49.78	59.08	75.90	91.86
eqV2	14.57	26.67	48.24	56.65	65.24	79.61	92.70
eSEN	18.63	30.91	51.50	59.39	67.40	80.79	93.04

Joint force magnitude-angle accuracy

Percentage of evaluated atoms satisfying both |Δ|F|| < threshold and Δθ < 1° or 20°. Each cell reports the Δθ < 1° fraction (left) and the Δθ < 20° fraction (right). Higher is better.

FP	< 0.01 eV/Å	< 0.02 eV/Å	< 0.05 eV/Å	< 0.07 eV/Å	< 0.1 eV/Å	< 0.2 eV/Å	< 0.5 eV/Å
Orb
SevenNet
SevenNet-0
SevenNet-Nano
MatterSim
MatterSim-1M
Nequix
MACE
CHGNet
GPTFF
M3GNet
UMA
ALIGNN
M3GNet-MatPES
TensorNet-MatPES
MACE-MatPES
NEP89
DPA4
DPA4-Air
GRACE
GRACE-1L
GRACE-2L
GRACE-2L-large
eqV2
eSEN

Large-force-error atoms

Percentage of evaluated atoms with |Δ|F|| exceeding each threshold. Lower is better.

FP	> 0.5 eV/Å	> 1 eV/Å	> 2 eV/Å	> 3 eV/Å	> 4 eV/Å	> 5 eV/Å	> 7 eV/Å	> 10 eV/Å
Orb	7.672	2.001	0.307	0.118	0.063	0.041	0.024	0.015
SevenNet	8.948	2.632	0.530	0.223	0.119	0.075	0.041	0.025
SevenNet-0	16.007	5.138	1.168	0.573	0.368	0.264	0.157	0.083
SevenNet-Nano	12.536	3.305	0.432	0.128	0.064	0.043	0.026	0.017
MatterSim	10.191	2.653	0.377	0.142	0.085	0.059	0.037	0.024
MatterSim-1M	11.130	2.851	0.492	0.184	0.104	0.069	0.039	0.025
Nequix	10.007	2.826	0.464	0.158	0.078	0.050	0.030	0.019
MACE	11.954	3.438	0.590	0.194	0.083	0.046	0.025	0.018
CHGNet	18.338	5.707	0.945	0.290	0.122	0.069	0.036	0.023
GPTFF	26.381	9.821	2.154	0.727	0.318	0.169	0.070	0.036
M3GNet	26.553	10.789	2.757	1.056	0.493	0.261	0.103	0.035
UMA	7.785	2.686	1.055	0.808	0.712	0.654	0.583	0.520
ALIGNN	41.519	24.231	12.236	8.083	6.027	4.793	3.366	2.304
M3GNet-MatPES	9.363	2.355	0.452	0.159	0.073	0.040	0.020	0.013
TensorNet-MatPES	6.080	1.196	0.175	0.065	0.043	0.033	0.024	0.018
MACE-MatPES	1.535	0.249	0.039	0.021	0.017	0.014	0.011	0.010
NEP89	22.997	7.603	1.730	0.633	0.318	0.205	0.127	0.085
DPA4	6.130	1.636	0.258	0.119	0.081	0.064	0.046	0.034
DPA4-Air	6.313	1.647	0.252	0.113	0.074	0.054	0.036	0.024
GRACE	7.349	2.076	0.284	0.107	0.061	0.042	0.027	0.019
GRACE-1L	12.517	3.849	0.742	0.273	0.127	0.074	0.044	0.029
GRACE-2L	9.983	3.061	0.811	0.485	0.352	0.275	0.187	0.118
GRACE-2L-large	8.139	2.329	0.452	0.193	0.103	0.062	0.029	0.017
eqV2	7.301	1.981	0.219	0.076	0.042	0.032	0.022	0.018
eSEN	6.959	1.928	0.249	0.101	0.062	0.046	0.032	0.022

Δ|F| MAE/RMSE across DFT force-magnitude subsets

Mean absolute error and root-mean-square error of Δ|F|, restricted to atoms with |F_DFT| above each threshold. The top row shows the average percentage of atoms (across FPs) above each threshold. Each cell reports MAE (left) and RMSE (right). MAE and RMSE use separate color scales; lower is better for both. FE atoms correspond to |F_DFT| > 1 eV/Å.

FP	All atoms	> 0.01 eV/Å	> 0.05 eV/Å	> 0.1 eV/Å	> 0.2 eV/Å	> 0.5 eV/Å	> 0.7 eV/Å	> 1 eV/Å	> 2 eV/Å
Atoms with \|F_DFT\| > threshold	100.0%	97.0%	86.0%	76.7%	66.8%	48.7%	38.7%	26.8%	8.3%
Orb
SevenNet
SevenNet-0
SevenNet-Nano
MatterSim
MatterSim-1M
Nequix
MACE
CHGNet
GPTFF
M3GNet
UMA
ALIGNN
M3GNet-MatPES
TensorNet-MatPES
MACE-MatPES
NEP89
DPA4
DPA4-Air
GRACE
GRACE-1L
GRACE-2L
GRACE-2L-large
eqV2
eSEN

FE relative force-magnitude error (r_F) thresholds

FE atoms correspond to |F_DFT| > 1 eV/Å. r_F = |Δ|F|| / |F_DFT|. Percentage of FE atoms with r_F below each threshold. Higher is better.

FP	<0.01	<0.05	<0.1	<0.2	<0.3	<0.4	<0.5	<1	<2
Orb	8.48	35.23	55.66	76.18	85.29	89.88	92.51	99.52	99.91
SevenNet	6.36	28.70	48.39	70.65	81.61	87.34	90.60	99.20	99.80
SevenNet-0	2.82	13.44	25.35	45.26	60.63	72.29	80.78	98.17	99.28
SevenNet-Nano	3.49	17.02	31.99	55.34	71.35	81.87	88.41	99.73	99.98
MatterSim	5.38	24.74	43.34	66.70	79.32	86.37	90.43	99.45	99.91
MatterSim-1M	4.75	22.41	40.15	63.73	77.31	85.39	90.35	99.30	99.84
Nequix	5.00	23.15	40.94	64.35	78.00	85.78	90.11	99.51	99.91
MACE	4.00	18.90	34.56	57.18	71.82	81.26	87.22	99.29	99.89
CHGNet	1.91	9.43	18.40	35.36	50.82	64.44	75.65	99.47	99.93
GPTFF	0.76	3.85	7.78	16.10	25.90	37.50	50.63	99.59	99.92
M3GNet	1.07	5.35	10.54	20.97	31.65	42.92	54.67	98.34	99.63
UMA	11.07	42.04	62.00	78.93	85.63	89.11	91.25	97.98	98.59
ALIGNN	0.58	2.84	5.67	11.57	17.92	24.98	32.99	81.91	87.76
M3GNet-MatPES	3.94	19.06	35.25	59.04	74.69	84.77	91.25	99.83	99.99
TensorNet-MatPES	5.22	24.87	45.18	71.85	85.78	92.88	96.41	99.92	100.00
MACE-MatPES	13.94	52.54	75.30	91.75	96.65	98.40	99.12	99.99	100.00
NEP89	2.12	10.44	20.58	39.30	55.17	68.11	78.23	98.08	99.69
DPA4	15.86	52.02	70.59	83.70	88.73	91.50	93.25	99.55	99.89
DPA4-Air	13.34	47.51	67.20	82.50	88.30	91.33	93.24	99.56	99.90
GRACE	10.69	41.74	62.53	80.06	87.01	90.51	92.59	99.64	99.94
GRACE-1L	3.36	16.38	31.04	54.13	70.14	80.57	87.03	99.33	99.89
GRACE-2L	5.00	23.51	41.87	65.65	78.47	85.46	89.36	98.59	99.31
GRACE-2L-large	7.34	32.11	52.70	74.20	83.81	88.69	91.47	99.36	99.85
eqV2	11.71	45.26	65.84	81.73	87.67	90.70	92.56	99.70	99.97
eSEN	13.37	48.16	67.73	82.37	87.88	90.76	92.55	99.63	99.94

Highly accurate force predictions

Percentage of evaluated atoms satisfying |Δ|F|| < threshold. Higher is better.

FP	< 0.01 eV/Å	< 0.02 eV/Å	< 0.05 eV/Å	< 0.07 eV/Å	< 0.1 eV/Å	< 0.2 eV/Å	< 0.5 eV/Å
M3GNet-MatPES-r2SCAN	5.93	11.61	26.34	34.42	44.42	65.71	88.81
TensorNet-MatPES-r2SCAN	6.39	12.45	28.17	36.78	47.53	70.16	92.17
MACE-MatPES-r2SCAN	12.82	23.81	46.83	56.78	67.22	84.61	96.91

Joint force magnitude-angle accuracy

Percentage of evaluated atoms satisfying both |Δ|F|| < threshold and Δθ < 1° or 20°. Each cell reports the Δθ < 1° fraction (left) and the Δθ < 20° fraction (right). Higher is better.

FP	< 0.01 eV/Å	< 0.02 eV/Å	< 0.05 eV/Å	< 0.07 eV/Å	< 0.1 eV/Å	< 0.2 eV/Å	< 0.5 eV/Å
M3GNet-MatPES-r2SCAN
TensorNet-MatPES-r2SCAN
MACE-MatPES-r2SCAN

Large-force-error atoms

Percentage of evaluated atoms with |Δ|F|| exceeding each threshold. Lower is better.

FP	> 0.5 eV/Å	> 1 eV/Å	> 2 eV/Å	> 3 eV/Å	> 4 eV/Å	> 5 eV/Å	> 7 eV/Å	> 10 eV/Å
M3GNet-MatPES-r2SCAN	11.189	2.866	0.538	0.189	0.085	0.043	0.018	0.009
TensorNet-MatPES-r2SCAN	7.834	1.636	0.254	0.097	0.058	0.041	0.024	0.014
MACE-MatPES-r2SCAN	3.091	0.543	0.087	0.042	0.028	0.021	0.012	0.007

Δ|F| MAE/RMSE across DFT force-magnitude subsets

FP	All atoms	> 0.01 eV/Å	> 0.05 eV/Å	> 0.1 eV/Å	> 0.2 eV/Å	> 0.5 eV/Å	> 0.7 eV/Å	> 1 eV/Å	> 2 eV/Å
Atoms with \|F_DFT\| > threshold	100.0%	96.4%	89.0%	84.1%	76.6%	57.8%	46.6%	33.3%	10.8%
M3GNet-MatPES-r2SCAN
TensorNet-MatPES-r2SCAN
MACE-MatPES-r2SCAN

FE relative force-magnitude error (r_F) thresholds

FE atoms correspond to |F_DFT| > 1 eV/Å. r_F = |Δ|F|| / |F_DFT|. Percentage of FE atoms with r_F below each threshold. Higher is better.

FP	<0.01	<0.05	<0.1	<0.2	<0.3	<0.4	<0.5	<1	<2
M3GNet-MatPES-r2SCAN	3.98	19.20	35.50	59.79	75.39	85.42	91.72	99.83	99.99
TensorNet-MatPES-r2SCAN	4.99	23.98	43.96	70.70	85.07	92.48	96.22	99.89	99.99
MACE-MatPES-r2SCAN	10.16	42.42	65.96	86.92	94.47	97.44	98.72	99.97	100.00

Highly accurate force predictions

Percentage of evaluated atoms satisfying |Δ|F|| < threshold. Higher is better.

FP	< 0.01 eV/Å	< 0.02 eV/Å	< 0.05 eV/Å	< 0.07 eV/Å	< 0.1 eV/Å	< 0.2 eV/Å	< 0.5 eV/Å
MACE	4.08	8.15	19.75	26.77	36.09	58.16	83.88
CHGNet	1.81	3.60	8.89	12.32	17.23	31.73	60.55
M3GNet	1.30	2.60	6.49	9.05	12.85	24.49	50.02
UMA	13.14	24.96	51.43	63.05	74.57	90.29	98.05
M3GNet-MatPES	2.30	4.59	11.30	15.63	21.87	39.76	70.26
TensorNet-MatPES	2.76	5.52	13.57	18.68	25.78	44.65	72.94
MACE-MatPES	4.12	8.19	19.62	26.34	35.07	55.22	79.65
Orb	9.07	17.65	38.85	49.42	61.28	81.47	95.12
SevenNet	7.68	15.08	34.19	44.29	56.09	77.56	93.83
SevenNet-0	2.03	4.08	10.11	13.99	19.59	35.56	63.32
SevenNet-Nano	4.61	9.20	22.14	29.88	40.03	63.02	87.55
MatterSim	4.21	8.36	20.40	27.90	38.00	62.41	89.04
MatterSim-1M	3.92	7.77	19.02	26.08	35.70	59.68	87.45
Nequix	5.88	11.64	27.25	36.03	46.91	69.29	90.02
GPTFF	1.18	2.33	5.85	8.15	11.61	22.64	48.84
NEP89	2.24	4.47	11.07	15.33	21.43	38.86	68.60
DPA4	16.64	31.00	60.23	71.54	81.69	93.44	98.47
DPA4-Air	14.63	27.66	55.55	67.16	78.16	92.08	98.33
GRACE	10.31	19.91	42.92	53.87	65.38	82.81	94.40
GRACE-1L	4.77	9.47	22.59	30.28	40.26	62.68	86.75
GRACE-2L	5.95	11.75	27.58	36.50	47.59	70.16	90.32
GRACE-2L-large	9.20	17.89	39.48	50.22	62.00	81.28	94.43
eqV2	10.10	19.47	41.92	52.66	64.24	82.00	94.03
eSEN	11.52	22.01	45.86	56.70	67.77	83.78	94.52

Joint force magnitude-angle accuracy

Percentage of evaluated atoms satisfying both |Δ|F|| < threshold and Δθ < 1° or 20°. Each cell reports the Δθ < 1° fraction (left) and the Δθ < 20° fraction (right). Higher is better.

FP	< 0.01 eV/Å	< 0.02 eV/Å	< 0.05 eV/Å	< 0.07 eV/Å	< 0.1 eV/Å	< 0.2 eV/Å	< 0.5 eV/Å
MACE
CHGNet
M3GNet
UMA
M3GNet-MatPES
TensorNet-MatPES
MACE-MatPES
Orb
SevenNet
SevenNet-0
SevenNet-Nano
MatterSim
MatterSim-1M
Nequix
GPTFF
NEP89
DPA4
DPA4-Air
GRACE
GRACE-1L
GRACE-2L
GRACE-2L-large
eqV2
eSEN

Large-force-error atoms

Percentage of evaluated atoms with |Δ|F|| exceeding each threshold. Lower is better.

FP	> 0.5 eV/Å	> 1 eV/Å	> 2 eV/Å	> 3 eV/Å	> 4 eV/Å	> 5 eV/Å	> 7 eV/Å	> 10 eV/Å
MACE	16.115	5.717	1.595	0.716	0.407	0.266	0.145	0.082
CHGNet	39.447	18.502	6.821	3.545	2.170	1.461	0.775	0.376
M3GNet	49.977	27.656	10.847	5.111	2.736	1.665	0.729	0.279
UMA	1.947	0.514	0.121	0.046	0.024	0.014	0.007	0.003
M3GNet-MatPES	29.737	12.432	4.042	1.927	1.117	0.719	0.374	0.191
TensorNet-MatPES	27.063	12.192	4.689	2.561	1.615	1.114	0.631	0.337
MACE-MatPES	20.351	8.939	3.191	1.582	0.938	0.620	0.328	0.172
Orb	4.879	1.472	0.415	0.198	0.122	0.083	0.041	0.021
SevenNet	6.168	1.667	0.346	0.124	0.065	0.039	0.020	0.010
SevenNet-0	36.677	18.945	7.994	4.461	2.859	1.983	1.087	0.561
SevenNet-Nano	12.448	3.665	0.788	0.303	0.165	0.104	0.055	0.030
MatterSim	10.965	3.171	0.876	0.423	0.262	0.180	0.106	0.062
MatterSim-1M	12.553	3.691	1.024	0.510	0.320	0.223	0.128	0.072
Nequix	9.981	3.213	0.880	0.418	0.256	0.178	0.102	0.058
GPTFF	51.158	27.599	11.181	5.839	3.482	2.252	1.098	0.475
NEP89	31.403	14.290	5.879	3.558	2.489	1.868	1.168	0.664
DPA4	1.525	0.456	0.130	0.064	0.045	0.035	0.027	0.021
DPA4-Air	1.674	0.480	0.130	0.065	0.045	0.036	0.026	0.020
GRACE	5.596	1.985	0.640	0.327	0.211	0.148	0.087	0.053
GRACE-1L	13.246	4.334	1.163	0.534	0.326	0.222	0.130	0.075
GRACE-2L	9.676	3.137	0.883	0.423	0.263	0.182	0.106	0.065
GRACE-2L-large	5.568	1.784	0.441	0.171	0.086	0.051	0.027	0.017
eqV2	5.973	2.131	0.674	0.343	0.219	0.157	0.092	0.054
eSEN	5.483	1.974	0.631	0.321	0.204	0.143	0.084	0.049

Δ|F| MAE/RMSE across DFT force-magnitude subsets

FP	All atoms	> 0.01 eV/Å	> 0.05 eV/Å	> 0.1 eV/Å	> 0.2 eV/Å	> 0.5 eV/Å	> 0.7 eV/Å	> 1 eV/Å	> 2 eV/Å
Atoms with \|F_DFT\| > threshold	100.0%	100.0%	100.0%	99.8%	99.0%	93.4%	88.2%	80.0%	56.6%
MACE
CHGNet
M3GNet
UMA
M3GNet-MatPES
TensorNet-MatPES
MACE-MatPES
Orb
SevenNet
SevenNet-0
SevenNet-Nano
MatterSim
MatterSim-1M
Nequix
GPTFF
NEP89
DPA4
DPA4-Air
GRACE
GRACE-1L
GRACE-2L
GRACE-2L-large
eqV2
eSEN

FE relative force-magnitude error (r_F) thresholds

FE atoms correspond to |F_DFT| > 1 eV/Å. r_F = |Δ|F|| / |F_DFT|. Percentage of FE atoms with r_F below each threshold. Higher is better.

FP	<0.01	<0.05	<0.1	<0.2	<0.3	<0.4	<0.5	<1	<2
MACE	9.79	44.27	70.76	90.48	96.12	98.14	99.03	99.93	100.00
CHGNet	3.81	18.72	36.13	63.60	80.63	89.88	94.69	99.65	99.98
M3GNet	3.04	14.97	29.03	52.11	68.17	78.78	85.60	97.18	99.70
UMA	36.87	83.78	94.47	98.38	99.22	99.57	99.74	99.97	100.00
M3GNet-MatPES	5.65	27.05	49.01	75.24	87.45	93.57	96.77	99.86	99.99
TensorNet-MatPES	6.26	29.94	53.08	78.36	89.23	94.42	97.12	99.87	99.98
MACE-MatPES	9.02	40.80	65.39	85.61	92.94	96.43	98.24	99.97	100.00
Orb	24.22	73.89	90.27	97.34	98.84	99.39	99.65	99.97	100.00
SevenNet	19.60	69.30	88.57	96.94	98.73	99.36	99.63	99.97	100.00
SevenNet-0	4.21	20.63	39.14	66.29	81.72	89.99	94.36	99.34	99.92
SevenNet-Nano	12.12	51.14	76.17	92.46	97.00	98.61	99.31	99.96	100.00
MatterSim	12.48	51.41	75.89	92.48	97.11	98.69	99.33	99.95	100.00
MatterSim-1M	11.54	48.71	73.52	91.26	96.49	98.36	99.15	99.92	99.98
Nequix	13.74	57.70	82.58	95.30	98.15	99.11	99.52	99.97	100.00
GPTFF	2.35	11.85	23.74	46.38	65.23	78.45	86.98	99.76	99.97
NEP89	5.36	25.51	46.00	71.78	84.89	91.71	95.43	99.77	99.98
DPA4	44.68	88.26	95.99	98.64	99.31	99.60	99.76	99.98	100.00
DPA4-Air	40.58	86.02	95.31	98.55	99.28	99.60	99.76	99.98	100.00
GRACE	25.59	76.04	91.71	97.53	98.83	99.35	99.61	99.97	100.00
GRACE-1L	11.78	50.12	75.52	92.20	96.85	98.55	99.27	99.95	100.00
GRACE-2L	14.41	58.73	82.77	95.22	98.10	99.08	99.51	99.96	100.00
GRACE-2L-large	23.64	73.47	90.80	97.58	98.96	99.46	99.69	99.97	100.00
eqV2	23.96	75.40	91.59	97.48	98.78	99.32	99.59	99.97	100.00
eSEN	27.77	78.07	92.51	97.64	98.84	99.35	99.61	99.97	100.00

Benchmark Datasets

MatPES-PBE

DFT functional: PBE

Equilibrium and off-equilibrium structures selected from finite-temperature MD trajectories, augmented with Materials Project ground-state structures.

Source: MatPES ↗

MatPES-r2SCAN

DFT functional: r2SCAN

The r2SCAN release of MatPES, evaluated against the three r2SCAN-trained FPs.

Source: MatPES ↗

OMat24 rattled-1000

DFT functional: PBE

The OMat24 rattled-1000 validation subset (approximately 117,000 structures rattled at a sampling temperature of 1000 K), evaluated against the FPs included in the FP-DeErr leaderboard.

Source: OMat24 ↗

Model versions and official sources for the evaluated FPs are documented once on the FP-DeErr home page.

Use FP-DeErr

Paired Cartesian DFT and FP forces
                  or
Full generator calculations on a dataset
                   ↓
        standardized force_results
                   ↓
          validation and analysis
                   ↓
       FP-DeErr force-error tables

Build standardized force results directly from paired Cartesian DFT and FP forces. Call build_force_results(dft_forces, fp_forces, structure_ids=None) from scripts/force_results.py with your own data.

Use a generator notebook as a template for full dataset/cluster generation. Register your FP's checkpoint and calculator setup in one of the three generator notebooks' POTENTIAL_REGISTRY, run the jobs on your cluster (the notebooks never submit jobs by themselves), then run the matching analysis notebook -- your FP appears in every table above.

git clone https://github.com/mogroupumd/FP-DeErr.git
cd FP-DeErr/Force_error
pip install -r requirements.txt
pip install jupyterlab
jupyter lab analysis/force_error_analysis_matpes_pbe.ipynb

See the Force_error README for the full quick start, the generator and analysis notebooks for complete operational detail, and the included Cartesian-force example for a small runnable slice of real data.

Add a potential

Interested in adding a foundation potential to FP-DeErr? Follow our Adding a Potential guide to evaluate your model, and visit Contribute for example outputs, results formats, and email submission instructions.

Force Prediction

Leaderboard

Leaderboard

Leaderboard

Error-decomposition metrics

Detailed results

Benchmark Datasets

Use FP-DeErr

Add a potential