holisticai.bias.metrics.avg_recall_ratio#

holisticai.bias.metrics.avg_recall_ratio(group_a, group_b, mat_pred, mat_true, top=None, thresh=0.5, normalize=False)[source]#

Average recall ratio

This function computes the ratio of average recall (over users) on minority and majority group.

Parameters

group_aarray-like: Group membership vector.
group_barray-like: Group membership vector.
mat_predmatrix-like: Matrix with shape (num_users, num_items). A recommender score (binary or soft pred) for each user,item interaction.
mat_truematrix-like: Matrix with shape (num_users, num_items). A target score (binary or soft pred) for each user,item pair.
topint, optional: If not None, the number of items that are shown to each user.
threshfloat, optional: Threshold indicating value at which a given item is shown to user (if top is None).
normalizebool, optional: If True, normalises the data matrix to [0,1] range.

Returns

float: Ratio of average recalls

Notes

\(\frac{\texttt{AVg_recall_min}}{\texttt{AVg_recall_maj}}\)

References

Examples

>>> import numpy as np
>>> from holisticai.bias.metrics import avg_recall_ratio
>>> group_a = np.array([1, 1, 1, 1, 0, 0, 0, 0, 0, 0])
>>> group_b = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1, 1])
>>> mat_pred = np.array([[0.9, 0.8, 0.4, 0.2],
                      [0.7, 0.9, 0.1, 0.7],
                      [0.3, 0.2, 0.3, 0.3],
                      [0.2, 0.1, 0.7, 0.8],
                      [0.8, 0.7, 0.9, 0.1],
                      [1. , 0.9, 0.3, 0.6],
                      [0.8, 0.9, 0.1, 0.1],
                      [0.2, 0.3, 0.1, 0.5],
                      [0.1, 0.2, 0.7, 0.7],
                      [0.2, 0.7, 0.1, 0.2]])
>>> mat_true = np.array([[0.7, 0.8, 0.4, 0.2],
                      [0.9, 0.9, 0.1, 0.2],
                      [0.3, 0.8, 0.2, 0.6],
                      [0.2, 0.1, 0.7, 0.8],
                      [0.6, 0.7, 0.9, 0.1],
                      [1. , 0.9, 0.3, 0.6],
                      [0.8, 0.1, 0.1, 0.1],
                      [0.2, 0.3, 0.1, 0.5],
                      [0.1, 0.2, 0.7, 0.7],
                      [0.2, 0.1, 0.1, 0.8]])
>>> avg_recall_ratio(
...     group_a, group_b, mat_pred, mat_true, top=2, thresh=0.5, normalize=False
... )
1.0