fatf.utils.metrics.tools.confusion_matrix_per_subgroup

fatf.utils.metrics.tools.confusion_matrix_per_subgroup(dataset: numpy.ndarray, ground_truth: numpy.ndarray, predictions: numpy.ndarray, column_index: Union[int, str], groupings: Optional[List[Union[float, Tuple[str]]]] = None, numerical_bins_number: int = 5, treat_as_categorical: Optional[bool] = None, labels: Optional[List[Union[float, str]]] = None) → Tuple[List[numpy.ndarray], List[str]]

Computes confusion matrices for every defined sub-population.

This is useful for computing a variety of performance metrics for each sub-population.

For warnings raised by this method please see the documentation of fatf.utils.data.tools.validate_indices_per_bin function.

Parameters

dataset, column_index, groupings, numerical_bins_number, and treat_as_categorical

These parameters are described in the documentation of fatf.utils.data.tools.group_by_column function and are used to define a grouping (i.e. sub-populations). If you have your own index-based grouping and would like to get sub-population-based confusion matrices, please consider using fatf.utils.metrics.tools.confusion_matrix_per_subgroup_indexed function.

ground_truth, predictions, and labels

These parameters are described in the documentation of fatf.utils.metrics.tools.get_confusion_matrix function and are used to calculate confusion matrices.

Returns

population_confusion_matrixList[numpy.ndarray]

A list of confusion matrices for each sub-population.

bin_namesList[strings]

The name of every sub-population (binning results) defined by the feature ranges for a numerical feature and feature value sets for a categorical feature.

Examples using fatf.utils.metrics.tools.confusion_matrix_per_subgroup

Measuring Fairness of a Predictive Model – Disparate Impact