holisticai.bias.metrics.avg_score_diff#

holisticai.bias.metrics.avg_score_diff(group_a, group_b, y_pred, q=0)[source]#

Average Score Difference

This function computes the difference in average scores between group_a and group_b.

If q is a vector, this function returns a vector with the respective result for each given quantile in q.

Interpretation

A value of 0 is desired. Negative values indicate the group_a has lower average score, so bias against group_a. Positive values indicate group_b has lower average score, so bias against group_b. Scale is relative to task.

Parameters

group_aarray-like: Group membership vector (binary)
group_barray-like: Group membership vector (binary)
y_predarray-like: Predictions vector (regression)
qfloat, array-like, optional: quantile of predictions considered, default=0.

Returns

float: Average Score Spread

Notes

:math:` exttt{AVgroup_a - AVgroup_b}`

Examples

>>> import numpy as np
>>> from holisticai.bias.metrics import avg_score_diff
>>> group_a = np.array([1, 1, 1, 1, 0, 0, 0, 0, 0, 0])
>>> group_b = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1, 1])
>>> y_pred = np.array([0.8, 0.9, 0.2, 0.1, 0.7, 0.9, 0.8, 0.6, 0.3, 0.5])
>>> avg_score_diff(group_a, group_b, y_pred)
-0.13333333333333341