Stability Prediction Benchmark

Correlation Matrix of Stability Metrics

engine correlation_matrix

config (1 key)

{
  "variables": [
    "formation_energy_per_atom",
    "energy_above_hull",
    "energy_prediction_mae",
    "force_field_mae"
  ]
}

expected results (2 keys)

{
  "energy_prediction_mae↔force_field_mae": {
    "direction": "positive",
    "note": "Model errors on energy and forces tend to co-vary"
  },
  "formation_energy_per_atom↔energy_above_hull": {
    "direction": "positive",
    "note": "Both measure thermodynamic stability but at different reference frames"
  }
}

Ridge Regression: Composition → Formation Energy

engine ridge_regression

config (3 keys)

{
  "predictors": [
    "crystal_graph_representation"
  ],
  "regularization": "l2",
  "target": "formation_energy_per_atom"
}

expected results (2 keys)

{
  "mae": {
    "max": 0.15,
    "units": "eV/atom"
  },
  "r_squared": {
    "min": 0.7,
    "note": "Linear baseline; composition alone captures bulk trends"
  }
}

Gradient Boosting: Full Features → Stability Classification

engine gradient_boosting←ridge_formation_energy

config (5 keys)

{
  "classification": true,
  "predictors": [
    "formation_energy_per_atom",
    "crystal_graph_representation",
    "neural_network_potential"
  ],
  "target": "energy_above_hull",
  "threshold": 0,
  "threshold_note": "energy_above_hull = 0 means on the hull (stable)"
}

expected results (2 keys)

{
  "accuracy": {
    "min": 0.8
  },
  "precision_stable_class": {
    "min": 0.75,
    "note": "Precision matters — false stable predictions waste synthesis effort"
  }
}

CHGNet Neural Potential Baseline

engine pydmclab_ml_potential_relaxation←ridge_formation_energy, gb_hull_classification

config (2 keys)

{
  "compare_to": [
    "ridge_formation_energy",
    "gb_hull_classification"
  ],
  "model": "chgnet"
}

expected results (2 keys)

{
  "energy_mae": {
    "max": 0.03,
    "note": "CHGNet typically achieves ~25-30 meV/atom",
    "units": "eV/atom"
  },
  "force_mae": {
    "max": 0.075,
    "units": "eV/Å"
  }
}