Bias Mitigation with Grid Search Reduction#

This tutorial demonstrates how to implement the “Grid search reduction” inprocessing method to enhance fairness in regression models using the holisticai library.

First, install the holisticai package if you haven’t already:

!pip install holisticai[all]

Then, import the necessary libraries.

[11]:

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
from holisticai.datasets import load_dataset
from holisticai.bias.mitigation import GridSearchReduction
from holisticai.bias.metrics import regression_bias_metrics

np.random.seed(0)
import warnings
warnings.filterwarnings("ignore")

Load the proprocessed “Communities and Crime” dataset.

[12]:

dataset = load_dataset('us_crime', protected_attribute="race")
dataset = dataset.train_test_split(test_size=0.2, random_state=0)
train_data = dataset['train']
test_data = dataset['test']

dataset

[12]:
[DatasetDict]
train [Dataset]
Instances: 1594
Features: X , y , p_attrs , group_a , group_b
Metadata: race: {'group_a': 'racePctWhite>0.5', 'group_b': 'racePctWhite<=0.5'}
test [Dataset]
Instances: 399
Features: X , y , p_attrs , group_a , group_b
Metadata: race: {'group_a': 'racePctWhite>0.5', 'group_b': 'racePctWhite<=0.5'}
[13]:

model = LinearRegression()
model.fit(train_data['X'], train_data['y'])
model

[13]:

LinearRegression()
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Bias Mitigation#

Traditional Implementation#

We will implement the “Grid search reduction” method, an in-processing technique to mitigate bias in the regression model.

[14]:

model = LinearRegression()
inprocessing_model = GridSearchReduction(constraints="BoundedGroupLoss",
                                         loss='Square', min_val=-0.1, max_val=0.1,
                                         grid_size=50).transform_estimator(model)

inprocessing_model.fit(train_data['X'], train_data['y'], train_data['group_a'], train_data['group_b'])
inprocessing_model

[14]:
[GridSearchReduction]
GridSearchReduction(constraints=BoundedGroupLoss, constraint_weight=0.5, loss=Square, min_val=-0.1, ...)
Type: Bias Mitigation Inprocessing
[15]:

y_pred = inprocessing_model.predict(test_data['X'])

df = regression_bias_metrics(
    test_data['group_a'],
    test_data['group_b'],
    y_pred,
    test_data['y'],
    metric_type='both'
)
df

[15]:
Value Reference
Metric
Disparate Impact Q90 0.025284 1
Disparate Impact Q80 0.111974 1
Disparate Impact Q50 0.412979 1
Statistical Parity Q50 -0.725221 0
No Disparate Impact Level 0.055754 -
Average Score Difference -0.375943 0
Average Score Ratio 0.308226 1
Z Score Difference -2.799448 0
Max Statistical Parity 0.784808 0
Statistical Parity AUC 0.454691 0
RMSE Ratio 0.600185 1
RMSE Ratio Q80 0.974341 1
MAE Ratio 0.474628 1
MAE Ratio Q80 0.963801 1
Correlation Difference 0.021459 0 
[16]:

grid_search_rmse = mean_squared_error(test_data['y'], y_pred, squared=False)
print("RMS error: {}".format(grid_search_rmse))

RMS error: 0.14223548828832497

Pipeline Implementation#

Implement the method using the pipeline.

[17]:

from holisticai.pipeline import Pipeline

inprocessing_model = GridSearchReduction(constraints="BoundedGroupLoss",
                                         loss='Square', min_val=-0.1, max_val=1.3,
                                         grid_size=20).transform_estimator(model)

pipeline = Pipeline(
    steps=[
        ("bm_inprocessing", inprocessing_model),
    ]
)

fit_params = {
    "bm__group_a": train_data['group_a'],
    "bm__group_b": train_data['group_b']
}

pipeline.fit(train_data['X'], train_data['y'], **fit_params)

predict_params = {
    "bm__group_a": test_data['group_a'],
    "bm__group_b": test_data['group_b'],
}
y_pred_pipeline = pipeline.predict(test_data['X'], **predict_params)
df_pipeline = regression_bias_metrics(
    test_data['group_a'],
    test_data['group_b'],
    y_pred,
    test_data['y'],
    metric_type='both'
)
df_pipeline

[17]:
Value Reference
Metric
Disparate Impact Q90 0.025284 1
Disparate Impact Q80 0.111974 1
Disparate Impact Q50 0.412979 1
Statistical Parity Q50 -0.725221 0
No Disparate Impact Level 0.055754 -
Average Score Difference -0.375943 0
Average Score Ratio 0.308226 1
Z Score Difference -2.799448 0
Max Statistical Parity 0.784808 0
Statistical Parity AUC 0.454691 0
RMSE Ratio 0.600185 1
RMSE Ratio Q80 0.974341 1
MAE Ratio 0.474628 1
MAE Ratio Q80 0.963801 1
Correlation Difference 0.021459 0 
[18]:

pipeline_rmse = mean_squared_error(test_data['y'], y_pred_pipeline, squared=False)
print("Pipeline RMSE: {}".format(pipeline_rmse))

Pipeline RMSE: 0.14977135111289078