holisticai.bias.metrics.aggregate_diversity

holisticai.bias.metrics.aggregate_diversity(mat_pred, top=None, thresh=0.5, normalize=False)

Aggregate Diversity

Given a matrix of scores, this function computes the recommended items foreach user, selecting either the highest-scored items or those above an inputthreshold. It then returns the aggregate diversity: the proportion of recommendeditems out of all possible items.

Interpretation

A value of 1 is desired. We wish for a high proportion of itemsto be shown to avoid the ‘rich get richer effect’.

Parameters

mat_predmatrix-like

Matrix with shape (num_users, num_items). A recommenderscore (binary or soft pred) for each user,item interaction.

topint, optional

If not None, the number of items recommended to each user.

threshfloat, optional

Threshold indicating value at which a given item is shown to user (if top is None).

normalizebool, optional

If True, normalises the data matrix to [0,1] range.

Returns

float

Aggregate Diversity

Notes

\(\frac{|Items\; shown|}{|Items|}\)

References

[1]

H Abdollahpouri and M Mansoury and R Burke and B Mobasher and E Malthouse (2021). User-centered Evaluation of Popularity Bias in Recommender Systems, ACM. <https://doi.org/10.1145%2F3450613.3456821>

Examples

>>> import numpy as np
>>> from holisticai.bias.metrics import aggregate_diversity
>>> mat_pred = np.array([[0.9, 0.8, 0.4, 0.2],
                      [0.7, 0.9, 0.1, 0.7],
                      [0.3, 0.2, 0.3, 0.3],
                      [0.2, 0.1, 0.7, 0.8],
                      [0.8, 0.7, 0.9, 0.1],
                      [1. , 0.9, 0.3, 0.6],
                      [0.8, 0.9, 0.1, 0.1],
                      [0.2, 0.3, 0.1, 0.5],
                      [0.1, 0.2, 0.7, 0.7],
                      [0.2, 0.7, 0.1, 0.2]])
>>> aggregate_diversity(mat_pred, top=None, thresh=0.8, normalize=True)
0.75