fatf.utils.metrics.subgroup_metrics.performance_per_subgroup

fatf.utils.metrics.subgroup_metrics.performance_per_subgroup(dataset: numpy.ndarray, ground_truth: numpy.ndarray, predictions: numpy.ndarray, column_index: Union[int, str], *args, label_index: int = 0, groupings: Optional[List[Union[float, Tuple[str]]]] = None, numerical_bins_number: int = 5, treat_as_categorical: Optional[bool] = None, labels: Optional[List[Union[float, str]]] = None, metric: Optional[str] = None, metric_function: Optional[Callable[[numpy.ndarray], float]] = None, **kwargs) → Tuple[List[float], List[str]]

Computes a chosen metric per sub-population for a data set.

This function combines fatf.utils.metrics.tools.confusion_matrix_per_subgroup function together with fatf.utils.metrics.subgroup_metrics.apply_metric (when using metric parameter) and fatf.utils.metrics.subgroup_metrics.apply_metric_function (when using metric_function parameter) functions. For the description of parameters, errors and exceptions please see the documentation of these functions.

Note

The metric_function parameter takes the precedence over the metric parameter is both are provided.

Returns

population_metricsList[numbers]

A list with the value of the selected metric for every sub-population.

bin_namesList[strings]

The name of every sub-population (binning results) defined by the feature ranges for a numerical feature and feature value sets for a categorical feature.

Examples using fatf.utils.metrics.subgroup_metrics.performance_per_subgroup

Measuring Robustness of a Predictive Model – Systematic Performance Bias