holisticai.bias.metrics.average_odds_diff

holisticai.bias.metrics.average_odds_diff(group_a, group_b, y_pred, y_true)

Average Odds Difference

This function computes the difference in average odds between group_a and group_b.

Interpretation

A value of 0 is desired. This metric ranges between -1 and 1, with negative values indicating bias against group_a, and positive values indicating bias against group_b. The range (-0.1,0.1) is considered acceptable.

Parameters

group_aarray-like

Group membership vector (binary)

group_barray-like

Group membership vector (binary)

y_predarray-like

Predictions vector (binary)

y_truearray-like

Target vector (binary)

Returns

float

AOD score

Notes

\(0.5 * (fpr_a-fpr_b + tpr_a-tpr_b)\)

Examples

>>> import numpy as np
>>> from holisticai.bias.metrics import average_odds_diff
>>> group_a = np.array([1, 1, 1, 1, 0, 0, 0, 0, 0, 0])
>>> group_b = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1, 1])
>>> y_pred = np.array([1, 1, 0, 0, 1, 1, 0, 1, 1, 1])
>>> y_true = np.array([1, 1, 0, 0, 1, 0, 1, 0, 0, 1])
>>> average_odds_diff(group_a, group_b, y_pred, y_true)
-0.3333333333333333