holisticai.bias.metrics.accuracy_matrix#

holisticai.bias.metrics.accuracy_matrix(p_attr, y_pred, y_true, groups=None, classes=None)[source]#

Multiclass Accuracy Matrix

Given a protected attribute and multiclass classification task, for each group and class this function computes the accuracy of predictions on that group for the one vs all classifier of that class.

Parameters

p_attrarray-like: Multiclass protected attribute vector.
y_predarray-like: Prediction vector (categorical).
y_truearray-like: Target vector (categorical).
groupslist, optional: Unique groups from p_attr in order
classeslist, optional: The unique output classes in order

Returns

pandas DataFrame: Accuracy Matrix : shape (num_groups, num_classes)

Examples

>>> import numpy as np
>>> from holisticai.bias.metrics import accuracy_matrix
>>> p_attr = np.array(["A", "A", "A", "A", "B", "B", "B", "B", "C", "C"])
>>> y_pred = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2, 0])
>>> y_true = np.array([0, 1, 1, 0, 1, 0, 2, 1, 2, 1])
>>> accuracy_matrix(p_attr, y_pred, y_true)
    0    1    2
A  1.0  0.5  0.0
B  0.0  1.0  0.0
C  0.0  0.0  1.0