holisticai.explainability.metrics#

Global Feature Importance

alpha_score

Alpha Score calculates the smallest proportion of features that account for the alpha percentage of the overall feature importance.

fluctuation_ratio

Calculate the fluctuation ratio for features based on partial dependencies.

spread_divergence

Calculates the spread divergence metric based on the inverse of the Jensen-Shannon distance (square root of the Jensen-Shannon divergence), for a given feature importance.

rank_alignment

Compute the rank alignment metric between conditional feature importance and ranked feature importance.

spread_ratio

The spread ratio, ranging from 0 to 1, measures the degree of evenness or concentration in the distribution of feature importance values.

Local Feature Importance

importance_stability

Determine the stability of feature importance.

rank_consistency

Calculate the rank consistency of local feature importances through all the instances.

Tree Based Metrics

tree_depth_variance

Compute the variance of the depths of the leaves in the tree (TDV).

weighted_average_depth

Weighted Average Depth calculates the average depth of a tree considering the number of samples that pass through each cut.

weighted_average_explainability_score

Weighted Average Explainability Score calculates the average depth of a tree considering the number of samples that pass through each cut.

weighted_tree_gini

Compute the weighted Gini index for the tree (WGNI).

tree_number_of_rules

Calculates the number of rules in a decision tree surrogate model.

tree_number_of_features

Calculates the number of features used in a decision tree surrogate model.

Surrogate Based Metrics

surrogate_features_stability

Calculate the stability of features used in a surrogate model.

surrogate_fidelity_classification

Calculate the surrogate fidelity for classification tasks.

surrogate_fidelity_regression

Calculate the surrogate fidelity for regression models.

surrogate_accuracy_degradation

Calculate the difference between the mean squared error of the original model and the surrogate model.

surrogate_mean_squared_error_degradation

Calculate the difference between the mean squared error of the original model and the surrogate model.