holisticai.bias.metrics.mad_score

holisticai.bias.metrics.mad_score(group_a, group_b, mat_pred, normalize=False)

Mean Absolute Deviation

Difference in average score for group_a and group_b.

Interpretation

A large value of MAD indicates differential treatment ofgroup a and group b. A positive value indicates thatgroup a received higher scores on average, whilea negative value indicates higher ratings for group b.

Parameters

group_aarray-like

Group membership vector.

group_barray-like

Group membership vector.

mat_predmatrix-like

Matrix with shape (num_users, num_items). A recommender score (binary or soft pred) for each user,item interaction.

normalizebool, optional

If True, normalises the data matrix to [0,1] range.

Returns

float

MAD Score

Notes

\(\texttt{avg_group_a - avg_group_b}\)

References

[1]

Ziwei Zhu, Xia Hu, and James Caverlee (2018). Fairness-Aware Tensor-Based Recommendation, <https://dl.acm.org/doi/pdf/10.1145/3269206.3271795>

[2]

Y Deldjoo, V W Anelli, H Zamani, A Bellogin, TDi, T D Noia (2021). A Flexible Framework for Evaluating User and Item Fairness in Recommender Systems, <https://link.springer.com/article/10.1007/s11257-020-09285-1>

[3]

Y Deldjooa, A Bellogin, T D Noiaa (2021). Explaining recommender systems fairness and accuracy through the lens of data characteristics, <https://www.sciencedirect.com/science/article/pii/S0306457321001503>

Examples

>>> import numpy as np
>>> from holisticai.bias.metrics import mad_score
>>> group_a = np.array([1, 1, 1, 1, 0, 0, 0, 0, 0, 0])
>>> group_b = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1, 1])
>>> mat_pred = np.array([[0.9, 0.8, 0.4, 0.2],
                        [0.7, 0.9, 0.1, 0.7],
                        [0.3, 0.2, 0.3, 0.3],
                        [0.2, 0.1, 0.7, 0.8],
                        [0.8, 0.7, 0.9, 0.1],
                        [1. , 0.9, 0.3, 0.6],
                        [0.8, 0.9, 0.1, 0.1],
                        [0.2, 0.3, 0.1, 0.5],
                        [0.1, 0.2, 0.7, 0.7],
                        [0.2, 0.7, 0.1, 0.2]])
>>> mad_score(group_a, group_b, mat_pred, normalize=False)
0.00833333333333336