MLIP Performance Explorer

Purpose

MLIPs commonly exhibit trade-offs among property errors. A model that performs better on one property may perform worse on another, so two such models cannot be ranked unambiguously without specifying which properties are more important.

For a fixed MLIP architecture, varying the training data, hyperparameters, and other training settings produces a family of candidate models. The per-family Pareto front identifies the nondominated models and forms an empirical performance envelope: the best combinations of property errors observed within the sampled training space. Although these models may not be fully optimized, their Pareto front provides evidence about the potential performance that may be attainable with that architecture and guidance for subsequent training and model selection.

High-dimensional Pareto fronts extend this evaluation to many properties simultaneously and help reveal whether low errors can be achieved jointly. However, as more properties are included, more models tend to become nondominated. Front membership alone therefore becomes less selective and should be interpreted together with benchmarks, error magnitudes, and other front-level comparisons.

Interpretation in the interactive app

• Per-family front: the observed performance envelope for one architecture under its sampled training process.
• Global front: the models that remain nondominated when all sampled architectures are compared directly.

Cross-family front comparisons are most informative when the model families use comparable datasets, evaluation properties, search effort, and sampling coverage. Otherwise, an observed front reflects the architecture together with its explored training pipeline, rather than the architecture alone.

Purpose

The Causality panel explores conditional-dependency relationships among MLIP property errors. Each node represents the prediction error or performance metric of one property. The PC-style algorithm tests whether pairs of property errors remain statistically related after conditioning on selected combinations of other errors. It then orients some remaining relationships when permitted by its statistical rules.

An undirected edge represents a relationship whose direction cannot be resolved by the analysis. A directed edge represents an orientation suggested by the algorithm under its assumptions; it does not prove that directly improving the source property will cause the target property to improve.

Because the graph is constructed from observational results of trained models, relationships may be produced by shared factors such as MLIP architecture, training-data composition, hyperparameters, computational budget, or model-selection procedures. Use the graph to generate hypotheses about related performance metrics and possible training strategies, then validate proposed causal relationships through controlled retraining experiments or other interventions.

Potential benefits

If validated by further studies, causal analysis may identify training choices that jointly improve multiple property errors, helping avoid blind or random optimization. It may also reduce model-development cost by narrowing the training search space and identifying property performances that can be improved simultaneously.

• Improving joint performance: identify training interventions that may simultaneously reduce errors in several properties.
• Reducing development cost: prioritize promising training settings, avoid unproductive search regions, or use inexpensive properties to screen models before costly evaluations.

Interpretation and limitations

• The nodes represent property errors or performance metrics, not the physical properties themselves. Lower standardized log-error means better performance.
• A directed edge proposes a direction, not the sign or magnitude of an effect. Reducing error A could reduce error B, indicating synergy, or increase error B, indicating a trade-off.
• The sign of a correlation or partial correlation is not automatically the sign of a causal effect. Direct and total effects require separate estimation with uncertainty intervals.
• Architecture, training data, loss weights, hyperparameters, random seeds, and model screening can be unmeasured common causes of several errors.
• PC interpretation requires assumptions including independent observations, causal Markov and faithfulness conditions, adequate conditional-independence tests, and no important hidden confounding or selection bias. These conditions have not been established for this model pool.
• Property errors are generally downstream outcomes. An actionable future graph should include controllable training variables as possible causes and property errors as outcomes.
• The browser limits conditioning sets to K ≤ 3. This truncated search is exploratory and is not guaranteed to recover a complete CPDAG.
• Results depend on significance level, conditioning limit, model filters, sample size, and selection process. Edge stability, false discoveries, and statistical power require evaluation.

Purpose

The Clustering panel groups MLIPs that exhibit similar joint performance across a selected set of properties. Each model is represented by its vector of standardized log-errors. Models with similar error vectors have similar multi-property performance profiles, even when they belong to different MLIP families or were produced using different training settings.

Clustering Pareto-optimal models is particularly useful for studying the empirical performance envelopes of MLIP architectures. It can reveal recurring performance regimes, identify architectures that produce similar error patterns, and help select representative candidates for further evaluation.

How the app constructs similarity components

For models i and j, the app calculates Euclidean distance over the selected standardized errors, then converts it to inverse-distance similarity:

Smaller distance means more similar performance. The app converts the selected similarity percentile into a cutoff, connects pairs at or above that cutoff, and defines clusters as the connected components of the resulting graph.

Model-source modes

• Pareto/Fam: uses the union of per-family Pareto fronts, preserving the observed performance envelope of every included architecture.
• Pareto/All: uses the global Pareto front and retains models that remain nondominated against all sampled families.
• All Models: uses the complete filtered population, including dominated, unstable, and poorly performing candidates.

What clusters can show

• A cluster dominated by one family may indicate a characteristic sampled error profile for that family.
• A mixed-family cluster shows that different architectures can reach similar combinations of property errors.
• Several clusters from the same family may reveal distinct performance regimes or trade-offs.
• Isolated or sparsely connected models may have unusual combinations of strengths and weaknesses.

Interpretation and limitations

• A cluster indicates similarity, not accuracy. Inspect the error values, benchmarks, and Pareto information before judging performance.
• Distance is dissimilarity; inverse distance is the similarity used for graph connections.
• Selected properties determine the scientific question. Correlated or duplicated properties can receive excessive combined weight.
• Euclidean distances can become less distinguishable in higher dimensions. Standardization aligns scales but does not eliminate distance concentration.
• Connected components have a chaining effect: A and C can share a cluster through B even without a direct A–C connection.
• The cutoff is a percentile of the current similarity distribution, not an absolute physical tolerance. Cluster stability should be checked over multiple thresholds.
• Unequal sampling of architectures and training settings can affect cluster composition and should not be interpreted as intrinsic capability.
• The DFT / Threshold reference is not an MLIP and should be excluded.
• Identical vectors have zero distance and should be treated as maximally similar through an explicit zero-distance policy.
• Centre gravity and node dragging affect only layout, not similarities or cluster membership.

Relation to the published workflow

The app displays thresholded connected components. The paper instead combines per-family Pareto-optimal models, sweeps approximately 80 similarity cutoffs, runs weighted Louvain community detection, and selects a cutoff using agreement with MLIP-family labels. The app's result is therefore best described as similarity components, not a reproduction of the published Louvain communities.