fatf.accountability.models.measures.systematic_performance_bias

fatf.accountability.models.measures.systematic_performance_bias(metrics_list: List[float], threshold: float = 0.8) → bool

Checks for a systematic bias in provided predictive performance values.

Please see the documentation of fatf.accountability.models.measures.systematic_performance_bias_grid function for a description of input parameters, errors and exceptions.

Note

This function expects a list of predictive performance per sub-group for tested data. To get this list please use either of the following functions: fatf.utils.metrics.subgroup_metrics.performance_per_subgroup/ fatf.utils.metrics.subgroup_metrics.performance_per_subgroup_indexed or fatf.utils.metrics.tools.confusion_matrix_per_subgroup/ fatf.utils.metrics.tools.confusion_matrix_per_subgroup_indexed in conjunction with fatf.utils.metrics.subgroup_metrics.apply_metric_function/ fatf.utils.metrics.subgroup_metrics.apply_metric.

Returns

is_biasedboolean

True if any sub-group pair has significantly (according to the specified threshold) different predictive performance, False otherwise.