holisticai.bias.metrics.recommender_rmse_ratio

holisticai.bias.metrics.recommender_rmse_ratio(group_a, group_b, mat_pred, mat_true, normalize=False)

Recommender RMSE ratio

This function computes the ratio of rmse between predictions and scores for group_a and group_b.

A value of 1 is desired. Lower values show bias against group_a. Higher values show bias against group_b.

group_aarray-like

Group membership vector.

group_barray-like

Group membership vector.

mat_predmatrix-like

Matrix with shape (num_users, num_items). A recommender score (binary or soft pred) for each user,item interaction.

mat_truematrix-like

Matrix with shape (num_users, num_items). A target score (binary or soft pred) for each user,item pair.

normalizebool, optional

If True, normalises the data matrix to [0,1] range.

float

Recommender RMSE ratio

:math:`

rac{ exttt{AVg_rmse_min}}{ exttt{AVg_rmse_maj}}`

>>> import numpy as np
>>> from holisticai.bias.metrics import recommender_rmse_ratio
>>> group_a = np.array([1, 1, 1, 1, 0, 0, 0, 0, 0, 0])
>>> group_b = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1, 1])
>>> mat_pred = np.array([[0.9, 0.8, 0.4, 0.2],
                      [0.7, 0.9, 0.1, 0.7],
                      [0.3, 0.2, 0.3, 0.3],
                      [0.2, 0.1, 0.7, 0.8],
                      [0.8, 0.7, 0.9, 0.1],
                      [1. , 0.9, 0.3, 0.6],
                      [0.8, 0.9, 0.1, 0.1],
                      [0.2, 0.3, 0.1, 0.5],
                      [0.1, 0.2, 0.7, 0.7],
                      [0.2, 0.7, 0.1, 0.2]])
>>> mat_true = np.array([[0.7, 0.8, 0.4, 0.2],
                      [0.9, 0.9, 0.1, 0.2],
                      [0.3, 0.8, 0.2, 0.6],
                      [0.2, 0.1, 0.7, 0.8],
                      [0.6, 0.7, 0.9, 0.1],
                      [1. , 0.9, 0.3, 0.6],
                      [0.8, 0.1, 0.1, 0.1],
                      [0.2, 0.3, 0.1, 0.5],
                      [0.1, 0.2, 0.7, 0.7],
                      [0.2, 0.1, 0.1, 0.8]])
>>> recommender_rmse_ratio(group_a, group_b, mat_pred, mat_true)
1.149630441384884