holisticai.bias.metrics.accuracy_diff

holisticai.bias.metrics.accuracy_diff(group_a, group_b, y_score, y_true)

Accuracy Difference

This function computes the difference in accuracy of predictions for group_a and group_b

Interpretation

A value of 0 is desired. This metric ranges between -1 and 1, with negative values indicating bias against group_a, and positive values indicating bias against group_b.

Parameters

group_aarray-like

Group membership vector (binary)

group_barray-like

Group membership vector (binary)

y_scorenumpy array

Probability estimates (regression)

y_truearray-like

Target vector (binary)

Returns

float

acc_diff : acc_a - acc_b

Examples

>>> import numpy as np
>>> from holisticai.bias.metrics import accuracy_diff
>>> group_a = np.array([1, 1, 1, 1, 0, 0, 0, 0, 0, 0])
>>> group_b = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1, 1])
>>> y_pred = np.array([1, 1, 0, 0, 1, 1, 0, 1, 1, 1])
>>> y_true = np.array([1, 1, 0, 0, 1, 0, 1, 0, 0, 1])
>>> accuracy_diff(group_a, group_b, y_pred, y_true)
0.6666666666666667