holisticai.bias.metrics.precision_matrix#

holisticai.bias.metrics.precision_matrix(p_attr, y_pred, y_true, groups=None, classes=None)[source]#

Multiclass Precision Matrix

Given a protected attribute and multiclass classification task, for each group and class this function computes the precision of predictions on that group for the one vs all classifier of that class.

Parameters

p_attrarray-like: Multiclass protected attribute vector.
y_predarray-like: Prediction vector (categorical).
y_truearray-like: Target vector (categorical).
groupslist, optional: Unique groups from p_attr in order
classeslist, optional: The unique output classes in order

Returns

pandas DataFrame: Precision Matrix : shape (num_groups, num_classes)

Examples

>>> import numpy as np
>>> from holisticai.bias.metrics import precision_matrix
>>> p_attr = np.array(["A", "A", "A", "A", "B", "B", "B", "B", "C", "C"])
>>> y_pred = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2, 0])
>>> y_true = np.array([0, 1, 1, 0, 1, 0, 2, 1, 2, 1])
>>> precision_matrix(p_attr, y_pred, y_true, groups=None, classes=None)
    0    1    2
A  1.0  0.5  NaN
B  0.0  1.0  0.0
C  NaN  0.0  1.0