holisticai.bias.metrics.abroca

holisticai.bias.metrics.abroca(group_a, group_b, y_score, y_true)

ABROCA (area between roc curves)

This function computes the area between the roc curve of group_a and the roc curve of group_b

Interpretation

A value of 0 is desired. This metric ranges between -1 and 1, with negative values indicating bias against group_a, and positive values indicating bias against group_b.

Parameters

group_aarray-like

Group membership vector (binary)

group_barray-like

Group membership vector (binary)

y_scorearray-like

Probability estimates (regression)

y_truearray-like

Target vector (binary)

Returns

float

ABROCA : roc_auc_a - roc_auc_b

Examples

>>> import numpy as np
>>> from holisticai.bias.metrics import abroca
>>> group_a = np.array([1] * 50 + [0] * 50)
>>> group_b = np.array([0] * 50 + [1] * 50)
>>> y_score = np.concatenate((np.linspace(0, 1, 50), np.linspace(0, 1, 50) ** 2))
>>> y_true = y_score + np.random.random(y_score.shape) > 0.5
>>> abroca(group_a, group_b, y_score, y_true)
0.11806478405315601