holisticai.bias.metrics.z_test_diff#

holisticai.bias.metrics.z_test_diff(group_a, group_b, y_pred)[source]#

Z Test (Difference)

This function computes the Z-test statistic for the difference in success rates. Also known as 2-SD Statistic.

Interpretation

A value of 0 is desired. This test considers the data unfair if the computed value is greater than 2 or smaller than -2, indicating a statistically significant difference in success rates.

Parameters

group_aarray-like: Group membership vector (binary)
group_barray-like: Group membership vector (binary)
y_predarray-like: Predictions vector (binary)

Returns

float: Z test (difference version)

Examples

>>> import numpy as np
>>> from holisticai.bias.metrics import z_test_diff
>>> group_a = np.array([1, 1, 1, 1, 0, 0, 0, 0, 0, 0])
>>> group_b = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1, 1])
>>> y_pred = np.array([1, 1, 1, 0, 1, 1, 0, 0, 0, 0])
>>> z_test_diff(group_a, group_b, y_pred)
1.290994449

References