holisticai.robustness.plots.plot_adp_and_adf#

holisticai.robustness.plots.plot_adp_and_adf(results_df)[source]#

Plots the Accuracy Degradation Profile (ADP) in a 2D plot, showing the percentage of samples above the threshold (ADP) on the vertical axis and dataset size (size_factor) on the horizontal axis, with the x-axis reversed.

Points are colored green if the model’s performance is acceptable (“OK”) and red if there is significant accuracy degradation (“acc degrad!”). The first point where performance degradation occurs is highlighted and circled (Accuracy Degradation Factor, ADF), and a vertical dotted line is drawn at this point to mark the corresponding dataset size.

Parameters

results_dfpd.DataFrame

A DataFrame containing the following columns: - ‘size_factor’ (float): Fraction of the dataset used in the evaluation. - ‘ADP’ (float): The percentage of samples above the threshold for acceptable accuracy. - ‘decision’ (str): A string indicating whether the model’s performance is acceptable

(‘OK’) or shows significant accuracy degradation (‘acc degrad!’).

  • ‘average_accuracy’ (float): The average accuracy across the samples.

  • ‘variance_accuracy’ (float): The variance of the accuracy across the samples.

Returns

None

This function does not return any values. It generates and displays a scatter plot with labeled points and a vertical line at the first accuracy degradation point.

Example

>>> import pandas as pd
>>> data = {
...     "size_factor": [0.95, 0.9, 0.85, 0.8, 0.75],
...     "ADP": [0.98, 0.97, 0.94, 0.87, 0.76],
...     "decision": ["OK", "OK", "OK", "acc degrad!", "acc degrad!"],
...     "average_accuracy": [0.97, 0.96, 0.93, 0.85, 0.74],
...     "variance_accuracy": [0.02, 0.03, 0.04, 0.05, 0.06],
... }
>>> results_df = pd.DataFrame(data)
>>> plot_adp_and_adf(results_df)

This will display a scatter plot with ‘size_factor’ on the x-axis (in reverse order) and ‘ADP’ on the y-axis. The first point where performance degrades (‘acc degrad!’) will be circled, and a vertical dotted line will be added to indicate the corresponding size factor.

Notes

  • The blue line represents the average accuracy with shaded areas indicating the variance. Green points indicate acceptable performance, while red points mark instances of degradation.

  • The first ‘acc degrad!’ point is highlighted with a red circle and a vertical dotted line to emphasize the Accuracy Degradation Factor (ADF).

  • The x-axis is inverted to show the dataset size decreasing from left to right.