fatf.utils.transparency.surrogate_evaluation
.local_fidelity_score¶

fatf.utils.transparency.surrogate_evaluation.
local_fidelity_score
(dataset: numpy.ndarray, data_row: Union[numpy.ndarray, numpy.void], global_predictive_function: Callable[numpy.ndarray, numpy.ndarray], local_predictive_function: Callable[numpy.ndarray, numpy.ndarray], metric_function: Callable[[numpy.ndarray, numpy.ndarray], float], explained_class_index: Optional[int] = None, explained_feature_indices: Optional[List[Union[str, int]]] = None, fidelity_radius_percentage: int = 5, samples_number: int = 50) → float[source]¶ Computes local fidelity between a global and a local (surrogate) model.
New in version 0.0.2.
For a selected data point (
data_row
), it samples uniformly around it within a hypersphere, which radius corresponds to a percentage – defined withfidelity_radius_percentage
parameter – of the maximum l2 distance between the specified data point and all the instances in thedataset
. (This sampling is based onfatf.utils.data.augmentation.LocalSphere
data augmenter.)Warning
A
dataset
with categorical features.This surrogate evaluation metric should not be used when the
dataset
contains categorical features (even when they are encoded, e.g., onehot encoding) since the l2 distance computed on mixed true numerical and (numericallyencoded) categorical features causes the local sample (computed with thefatf.utils.data.augmentation.LocalSphere
data augmenter) to be illdefined. Feature scaling could possibly be used to overcome this issue, however we leave such consideration up to the user.The global and local predictive functions can be either: a probabilistic predictor, a (multiclass) classifier or a regressor.
Global Model
Local Model
probabilistic
classifier
regressor
probabilistic
OK, e.g., KLdivergence
OK, e.g., logloss
Not possible
classifier
OK (via thresholding)
OK
Not possible
regressor
OK for a single class
Not possible
OK
If the
global_predictive_function
outputs probabilities, the following should be considered for different types of a local model:The local model is probabilistic as well:
a native probabilistic evaluation metric, such as the Kullback–Leibler divergence, can be used; or
a thresholding can be applied or a top prediction can be chosen for both the local and the global probabilistic prediction and a classic classification performance metric can be used.
The local model is a classifier – the probabilistic output of the global model has to be thresholded or the top prediction needs to be selected and a classic classification performance metric can be used.
The local model is a regressor – this is only possible if the regressor is fitted for the probabilistic output of one of the classes. In this case any of the standard regression evaluation measures can be used.
If the
global_predictive_function
is a classifier, the following should be considered for different types of a local model:The local model is probabilistic:
a native performance metric, like logloss, can be used; or
the probabilistic output of the local predictor can be thresholded or the top label selected and compare using standard classification performance metrics.
The local model is a classifier as well – any standard (multiclass) classification performance metric can be used.
Having a local regressor is not possible in this case.
Finally, if the
global_predictive_function
is a regressor, the local model can only be a regressor as well, in which case any standard regression evaluation metric can be used.If the problem being modelled is multiclass (for probabilistic models and classifiers), the local model can either be fitted to the original multiclass problem or as onevstherest for a selected class. In the latter case, when the global model is probabilistic, the
explained_class_index
parameter may be used to specify the class (column index) that thedata_row
belongs to (according to the global model) – in this case only the selected column with probabilities will be passed to the local fidelity score (metric_function
) function.Note
Why to train the local model as onevstherest?
When the local model is trained in the same output domain as the global model, the explanations extracted from this local model apply to all of the possible classes, what for some types of local models renders them uninformative. For example, consider training a decision tree locally and using the feature importance it provides. In this case we know which features are important in this local space but we cannot attribute these importances to any of the possible classes. However, a different type of explanation extracted from the same tree, for example, the logical conditions extracted from a roottoleaf path that the selected
data_row
falls into, can be perfectly reasonable.If, on the other hand, the local model is trained as onevsthe rest, where the “one” class is often set to be the class of the selected
data_row
, any type of the explanation can be attributed to the selected class. In this case feature importances extracted from the local model can attributed to the selected class in the specified neighbourhood. This mode of training the local model is required when the global model is probabilistic and the local one is a regressor, and optional for all the other combinations of the two.The consequence of training the local model as onevstherest is the need for train a separate local model for every class desired to be explained. For some local models and explanation types this is a requirement. For example, when the local model is a linear regression (trained on probabilities of a selected class) the only possible explanation is feature importance, which is meaningless in other cases.
In general, when evaluating the quality of a local surrogate, the most truthful measure would be the one achieved when the local model is trained on the same set of target classes. A good quality of a local onevstherest model with respect to the global model should be treated with caution as it only indicates that the local model excels at this task and may not be a good approximation of the global decisive process at all. Comparing quality of two local models where one is multiclass and the other onevstherest is relatively complex and should be done with caution (the former local model has a more difficult task to solve).
Examples of how to define the
metric_function
can be found in the Examples section down below. This local fidelity evaluation is inspired by the local fidelity method introduced in [LAUGEL2018SPHERES]. LAUGEL2018SPHERES
Laugel, T., Renard, X., Lesot, M. J., Marsala, C., & Detyniecki, M. (2018). Defining locality for surrogates in posthoc interpretablity. Workshop on Human Interpretability for Machine Learning (WHI) – International Conference on Machine Learning, 2018.
 Parameters
 datasetnumpy.ndarray
A 2dimensional numpy array with a dataset used to initialise the data sampler.
 data_rowUnion[numpy.ndarray, numpy.void]
A data point around which local fidelity is evaluated.
 global_predictive_functionCallable[[np.ndarray], np.ndarray]
A Python callable (e.g., a function) that is responsible for predicting data points in the global model. This function can either be probabilistic, i.e., return a 2dimensional numpy array with probabilities for every possible target class; a regressor (returning a 1dimensional regression values array) or a classifier (returning a 1dimensional class prediction array). Regardless of the type it must allow only one required parameter – a 2dimensional data array to be predicted.
 local_predictive_functionCallable[[np.ndarray], np.ndarray]
A Python callable (e.g., a function) that is responsible for predicting data points in the local (surrogate) model. For more details about the allowed function types please see the description of the
global_predictive_function
parameter. metric_functionCallable[[numpy.ndarray, numpy.ndarray], float]
A Python callable (e.g., a function) that computes a (performance) metric between the predictions of the global model (
global_predictive_function
) and the predictions of the local (surrogate) model (local_predictive_function
). The passed callable object must take exactly two required parameters: the first one being predictions of the global model and the latter predictions of the local model, and return a number (float) representing performance comparison of the two. This callable object has to be adjusted to the types of global and local predictive functions. explained_class_indexinteger, optional (default=None)
If the global model (
global_predictive_function
) is probabilistic, this parameter allows to select a single column of probabilities for a selected class to be passed to themetric_function
. This parameter is useful when the local (surrogate) model is a regressor predicting probabilities of this chosen class (the class being explained). explained_feature_indicesList[IndexType], optional (default=None)
If the local (surrogate) model was trained on a subset of the features, this parameter allows to indicate which features should be used when predicting the generated data with the local model. If
None
, all of the features will be used. fidelity_radius_percentageinteger, optional (default=5)
The locality of the fidelity measure is enforced by limiting the distance from the selected
data_row
to generated data, which is used for fidelity metric evaluation. This radius (of a hypersphere around the selecteddata_row
) is defined as a percentage of the largest l2 distance between any two data points in the inputdataset
within which the evaluation data will be sampled. samples_numberinteger, optional (default=50)
The number of samples to be generated when computing the local fidelity score.
 Returns
 fidelity_scorefloat
A metric of “closeness” between the global and the local predictive function predictions calculated using the
metric_function
on the sampled data.
 Raises
 IncompatibleModelError
The
global_predictive_function
or thelocal_predictive_function
does not required exactly one parameter. IncorrectShapeError
The input
dataset
is not a 2dimensional numpy array. The inputdata_row
is not 1dimensional: either a 1dimensional numpy array or a numpy void object for structured rows. The number of columns (features) in thedata_row
is different to the number of columns in the inputdataset
. IndexError
Some of the
explained_feature_indices
are invalid column indices for the inputdataset
. TypeError
The input
dataset
is not of a base type. The dtype of thedata_row
is too different from the dtype of thedataset
. Theglobal_predictive_function
or thelocal_predictive_function
is not a Python callable. Themetric_function
is not a Python callable or it does not require exactly two parameters. Theexplained_class_index
is neitherNone
nor an integer. Theexplained_feature_indices
is neitherNone
nor a Python list. Thefidelity_radius_percentage
is not an integer. Thesamples_number
is not an integer. ValueError
The
explained_class_index
is a negative integer or out of bounds for the number of classes outputted by the global probabilistic model (global_predictive_function
). Thefidelity_radius_percentage
is smaller than 1 or larger than 100. Thesamples_number
is smaller than 1.
 Warns
 UserWarning
If the user specifies the
explained_class_index
parameter for a global model that is not probabilistic, this parameter is ignored, about what the user is warned.
Examples
The metric function should be adjusted to the type of the global and local predictors (and the use of the
explained_class_index
parameter).>>> import numpy as np >>> data = np.array([[0, 1], [1, 1], [1, 0]]) >>> targets = np.array(['a', 'b', 'c'])
Let us assume that the global model is probabilistic, the local model is a regressor and we are explaining class
'b'
with index1
. (The index of the class is based on the lexicographical ordering of all the unique target values.)>>> explained_class_index = 1
>>> import fatf.utils.models.models as fatf_models >>> global_model = fatf_models.KNN(k=1) >>> global_model.fit(data, targets)
>>> probabilities = global_model.predict_proba(data) >>> selected_class_probabilities = probabilities[:, explained_class_index]
>>> local_model = fatf_models.KNN(k=1, mode='regressor') >>> local_model.fit(data, selected_class_probabilities)
One way to evaluate the performance of our local (surrogate) model in this scenario is the Mean Squared Error:
>>> def mse(global_predictions, local_predictions): ... mse = np.square(global_predictions  local_predictions) ... mse = mse.mean() ... return mse
>>> import fatf.utils.transparency.surrogate_evaluation as surrogate_eval >>> mse_fidelity_score = surrogate_eval.local_fidelity_score( ... data, data[0], global_model.predict_proba, local_model.predict, ... mse, explained_class_index=explained_class_index) >>> mse_fidelity_score 0.0
Alternatively, if
scikitlearn
is available, an ROC can be computed, in which case the probabilities of the selected class need to be thresholded:>>> import sklearn.metrics >>> def roc(global_predictions, local_predictions): ... global_predictions[global_predictions >= .5] = 1 ... global_predictions[global_predictions < .5] = 0 ... global_predictions = global_predictions.astype(int) ... ... roc = sklearn.metrics.roc_auc_score(global_predictions, ... local_predictions) ... return roc
>>> roc_fidelity_score = surrogate_eval.local_fidelity_score( ... data, data[1], global_model.predict_proba, local_model.predict, ... roc, explained_class_index=explained_class_index) >>> roc_fidelity_score 1.0
If both models are classifiers trained with the same set of target classes,
>>> local_classifier = fatf_models.KNN(k=1) >>> local_classifier.fit(data, targets)
a simple accuracy (implemented in FAT Forensics) can be used:
>>> import fatf.utils.metrics.metrics as fatf_metrics >>> import fatf.utils.metrics.tools as fatf_metrics_tools >>> def accuracy(global_predictions, local_predictions): ... confusion_matrix = fatf_metrics_tools.get_confusion_matrix( ... local_predictions, global_predictions, labels=['a', 'b', 'c']) ... accuracy = fatf_metrics.accuracy(confusion_matrix) ... return accuracy
>>> accuracy_fidelity_score = surrogate_eval.local_fidelity_score( ... data, data[2], global_model.predict, local_classifier.predict, ... accuracy) >>> accuracy_fidelity_score 1.0
(Note
global_model.predict
instead of`global_model.predict_proba
.)