API Reference (0.1.2)¶
This is the class and function reference of FAT Forensics. Please refer to the full user guide for further details, as the class and function raw specifications may not be enough to give full guidelines on their uses.
Note
The package is designed to work with both classic and structured numpy arrays. The latter is introduced to help manage numpy arrays holding vanila categorical features. Please see the Measuring Fairness of a Data Set and Measuring Fairness of a Predictive Model – Disparate Impact examples to see how the package can be used with a structured numpy array.
FAT Forensics¶
FAT Forensics is a Python module integrating a variety of fairness, accountability (security, privacy) and transparency (explainability, interpretability) approaches to assess social impact of artificial intelligence systems.
fatf.fairness: Fairness¶
The fatf.fairness module implements a variety of fairness algorithms.
This module holds a variety of techniques that can be used to assess
fairness of artificial intelligence pipelines and the machine
learning process: data (fatf.fairness.data), models
(fatf.fairness.models) and predictions
(fatf.fairness.predictions).
fatf.fairness.data: Fairness for Data¶
The fatf.fairness.data module implements fairness algorithms for data.
Checks for systemic bias in a dataset. |
|
Indicates whether a dataset has a systemic bias. |
fatf.fairness.models: Fairness for Models¶
The fatf.fairness.models module holds fairness algorithms for models.
Calculates selected disparate impact grid for a data set. |
|
Calculates selected disparate impact grid for indexed data. |
|
Checks if any sub-population pair violates chosen disparate impact measure. |
|
Checks for demographic parity between all of the sub-populations. |
|
Checks for equal opportunity between all of the sub-populations. |
|
Checks if accuracy difference of all grouping pairs is within tolerance. |
fatf.fairness.predictions: Fairness for Predictions¶
The fatf.fairness.predictions module houses fairness for predictions.
Checks counterfactual fairness of a prediction given a model. |
|
Checks for counterfactual fairness using a counterfactual fairness arrays. |
fatf.accountability: Accountability¶
The fatf.accountability module holds a range of accountability methods.
This module holds a variety of techniques that can be used to assess privacy, security and robustness of artificial intelligence pipelines and the machine learning process: data, models and predictions.
fatf.accountability.data: Accountability for Data¶
The fatf.accountability.data module implements accountability algorithms
for data.
Computes information needed for evaluating and remedying sampling bias. |
|
Computes information needed for evaluating and remedying sampling bias. |
|
Checks for a pairwise sampling bias based on the provided threshold. |
|
Checks for a pairwise sampling bias based on the provided threshold. |
fatf.accountability.models: Accountability for Models¶
The fatf.accountability.models module holds accountability algorithms
for models.
Checks for a systematic bias in provided predictive performance values. |
|
Checks for pairwise systematic bias in group-wise predictive performance. |
fatf.transparency: Transparency¶
The fatf.transparency module implements various transparency algorithms.
This module holds a variety of techniques that can be used to assess explainability and interpretability of artificial intelligence pipelines and the machine learning process: data, models and predictions.
fatf.transparency.data: Transparency for Data¶
The fatf.transparency.data module holds data transparency algorithms.
Describes categorical (textual) and numerical columns in the input array. |
|
Describes a numerical numpy array with basic statistics. |
|
Describes a categorical numpy array with basic statistics. |
fatf.transparency.models: Transparency for Models¶
The fatf.transparency.models module holds models transparency functions.
fatf.transparency.models.feature_influence: Feature Influence¶
The fatf.transparency.models.feature_influence module holds functions
for calculating feature influence for predictive models.
This module implements Partial Dependence (PD) and Individual Conditional Expectation (ICE) – model agnostic feature influence measurements.
Calculates Individual Conditional Expectation for a selected feature. |
|
Merges multiple Individual Conditional Expectation arrays. |
|
Calculates Partial Dependence based on Individual Conditional Expectations. |
|
Calculates Partial Dependence for a selected feature. |
fatf.transparency.models.submodular_pick: Submodular Pick¶
The fatf.transparency.models.submodular_pick module implements the
submodular pick algorithm proposed by [Ra376fa965786-RIBEIRO2016WHY].
New in version 0.1.1.
- Ra376fa965786-RIBEIRO2016WHY
Ribeiro, M.T., Singh, S. and Guestrin, C., 2016, August. Why should I trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1135-1144). ACM.
Applies submodular pick to explanations of a given subset of data. |
fatf.transparency.predictions: Transparency for Predictions¶
The fatf.transparency.predictions module holds predictions transparency
functions.
fatf.transparency.predictions.counterfactuals: Counterfactual Explainers¶
The fatf.transparency.predictions.counterfactuals module implements
counterfactual explainers for predictions.
Generates counterfactual explanations of black-box classifier predictions. |
Translates a counterfactuals array into a textual description. |
fatf.transparency.predictions.surrogate_explainers: Surrogate Explainers¶
New in version 0.0.2.
The fatf.transparency.predictions.surrogate_explainers module implements
example surrogate explainers.
Guidelines and tips for building custom surrogate explainers for various types of data (tabular, image and text) can be found in the How-to Guides part of the documentation:
The scikit-learn package is required for the surrogate tree and linear
model (LIME) explainers to work. When importing this module with
scikit-learn missing, the user will only be warned about disabling this
functionality not being available and the two explainers will
be disabled to allow using the abstract SurrogateTabularExplainer without
the scikit-learn package installed.
An abstract parent class for implementing surrogate explainers. |
|
A tabular LIME explainer – a surrogate explainer based on a linear model. |
|
A surrogate explainer based on a decision tree. |
fatf.transparency.predictions.surrogate_image_explainers: Surrogate Image Explainers¶
The fatf.transparency.predictions.surrogate_image_explainers module
implements a bLIMEy version of the LIME surrogate image explainer.
New in version 0.1.1.
The scikit-learn, scikit-image and Pillow packages are required for
the surrogate image explainer to work.
Implements a surrogate image explainer equivalent to LIME. |
fatf.transparency.sklearn: Scikit-learn Explainers¶
The fatf.transparency.sklearn module implements scikit-learn explainers.
This module requires the scikit-learn package to be installed.
fatf.transparency.sklearn.tools: Scikit-learn Explainer Tools¶
New in version 0.0.2.
The fatf.transparency.sklearn.tools module implements a base
scikit-learn explainer.
Implements a base scikit-learn model explainer class. |
Checks whether a class instance or a class is a scikit-learn predictor. |
|
Checks whether a class instance (object) is a scikit-learn predictor. |
fatf.transparency.sklearn.linear_model: Linear Model Explainers¶
New in version 0.0.2.
The fatf.transparency.sklearn.linear_model module implements linear
scikit-learn model explainers.
A scikit-learn linear model explainer class. |
Extracts coefficients (feature importances) of a linear scikit-learn model. |
fatf.vis: Visualisations¶
The fatf.vis module implements visualisations of various FAT methods.
This module requires the matplotlib package to be installed.
Plots Individual Conditional Expectation for a selected class. |
|
Plots Partial Dependence for a selected class. |
|
Plots an importance-based surrogate explanation, e.g., LIME. |
fatf.exceptions: Exceptions, Errors and Warnings¶
The fatf.exceptions module holds custom exceptions, errors and warnings.
Base class for FAT Forensics exceptions (inherits from |
|
Exception raised when the shape of an array is not what is expected. |
|
Exception raised when an explainer lacks desired functionality. |
|
Exception raised when a model lacks desired functionality. |
|
Exception raised when a model is unfitted and a fitted one is expected. |
|
Exception raised when a model is fitted and an unfitted one is expected. |
fatf.utils: Utilities¶
The fatf.utils module includes various utilities.
Developer guide: See the Developers Guide page for further details.
Base classes¶
An abstract class for implementing data augmentation methods. |
|
An abstract class used to implement predictive models. |
fatf.utils.array: Array Utilities¶
The fatf.utils.array module holds array tools and validation functions.
Identifies indices of columns with numerical and non-numerical values. |
|
Returns a numpy array with column indices that the input array is missing. |
|
Checks whether all the input |
|
Finds the more general type of the two given. |
|
Calls either local or numpy’s |
|
Calls either local or numpy’s |
|
Converts an array like object into an unstructured array. |
|
Determines whether a numpy dtype object is of numerical type. |
|
Determines whether a numpy dtype object is of textual type. |
|
Determines whether a numpy dtype object is one of base types. |
|
Determines whether a numpy dtype object is flat. |
|
Checks whether two numpy dtypes are similar. |
|
Determines whether two numpy array-like object have a similar data type. |
|
Determines whether a numpy array-like object has a numerical data type. |
|
Determines whether a numpy array-like object has a textual data type. |
|
Determines whether a numpy array-like object holds base data types. |
|
Determines whether a numpy array-like object is 1-dimensional. |
|
Determines whether a numpy array-like object has 2 dimensions. |
|
Determines whether the input is a structured numpy array’s row object. |
|
Checks if the input is either a 1D numpy array or a structured numpy row. |
|
Determines whether a numpy array-like object is a structured array. |
fatf.utils.data: Data Utilities¶
The fatf.utils.data module holds data tools and data sets.
fatf.utils.data.datasets: Data Sets¶
The fatf.utils.data.datasets module holds examples of data sets.
The iris data set is returned as a classic numpy array, whereas the health records data set is a structured numpy array.
Loads a dataset from a file. |
|
Loads in a fake health records dataset. |
|
Loads the IRIS dataset [R47dce3bbc9de-FISHER1936]. |
fatf.utils.data.tools: Data Set Tools¶
The fatf.utils.data.tools module implements tools for data set handling.
Groups row indices of an array based on value grouping of a chosen column. |
|
Applies a function to the specified groups of labels and predictions. |
|
Validates a list of binned indices. |
|
Validates a binary, square and symmetric numpy array. |
fatf.utils.data.augmentation: Data Set Augmentation¶
The fatf.utils.data.augmentation module implements data set augmenters.
An abstract class for implementing data augmentation methods. |
|
Sampling data from a normal distribution. |
|
Sampling data from a truncated normal distribution. |
|
Sampling data with the Mixup method. |
|
Sampling data to discover instances spanning all the possible classes. |
|
Sampling data in a hyper-sphere around the closest decision boundary. |
|
Sampling data in a hyper-sphere around the selected data point. |
fatf.utils.data.instance_augmentation: Data Point Augmentation¶
New in version 0.0.2.
The fatf.utils.data.instance_augmentation module implements various
augmentation function for 1-dimensional numpy array-like objects.
Changed in version 0.1.1: Added the random_binary_sampler function.
Samples non-zero elements of the binary |
|
Samples |
fatf.utils.data.transformation: Data Set Transformation¶
New in version 0.0.2.
The fatf.utils.data.transformation module holds data transformation
functions.
Creates a binary representation of the |
fatf.utils.data.density: Data Set Density Checking and Estimation¶
The fatf.utils.data.density module implements data density estimators.
Checks and scores density in the |
fatf.utils.data.discretisation: Data Set Discretisation¶
New in version 0.0.2.
The fatf.utils.data.discretisation module implements data
discretisation approaches.
An abstract class that all discretiser implementations should inherit from. |
|
Discretises selected numerical features of the |
fatf.utils.data.feature_selection: Data Set Feature Selection¶
New in version 0.0.2.
The fatf.utils.data.feature_selection module holds feature selection
tools.
fatf.utils.data.feature_selection.sklearn: sklearn Feature Selection¶
New in version 0.0.2.
The fatf.utils.data.feature_selection.sklearn module implements
scikit-learn-based feature selection approaches.
Selects the specified number of features based on Lasso path coefficients. |
|
Selects the specified number of features based on iterative importance. |
|
Selects the specified number of features based on their absolute weight. |
fatf.utils.data.occlusion: Image Occlusion¶
The fatf.utils.data.occlusion module implements functions to generate
partial occlusion of images represented as numpy arrays.
New in version 0.1.1.
Implements functionality for (partiall) occlusion a segmented image. |
fatf.utils.data.segmentation: Image Segmentation¶
The fatf.utils.data.segmentation module implements image segmenters.
New in version 0.1.1.
An abstract class implementing image segmentation functionality. |
|
Wraps the slic segmentation algorithm implemented in scikit-image. |
|
Wraps the quickshift segmentation algorithm implemented in scikit-image. |
Generates a boolean mask for pixels belonging to the specified segments. |
fatf.utils.models: Models Utilities¶
The fatf.utils.models module holds models tools and utilities.
fatf.utils.models.models: Example Models¶
The fatf.utils.models.models module holds custom models.
The models implemented in this module are mainly used for used for FAT Forensics package testing and the examples in the documentation.
A K-Nearest Neighbours model based on Euclidean distance. |
fatf.utils.models.processing: Model Processing¶
The fatf.utils.models.processing module implements model processing
functions.
New in version 0.1.1.
Slices |
fatf.utils.models.validation: Model Validation Tools¶
The fatf.utils.models.validation module validates models functionality.
This module holds functions responsible for validating models functionality across the FAT Forensics package.
Checks whether a model object has all the required functionality. |
fatf.utils.distances: Distances and Distance Utilities¶
The fatf.utils.distances module holds a variety of distance metrics.
The distance metrics and tools implemented in this module are mainly used for
the fatf.utils.models.models.KNN model implementation, to measure
distance (and similarity) of data points for various functions in this package
as well as for documentation examples and testing.
Computes a distance matrix (2-D) between all rows of the |
|
Computes the distance between a data point and an array of data. |
|
Calculates the Euclidean distance between two 1-dimensional numpy “arrays”. |
|
Calculates the Euclidean distance between |
|
Calculates the Euclidean distance matrix between rows in |
|
Calculates the Hamming distance between two strings |
|
Computes the Hamming distance between 1-dimensional non-numerical arrays. |
|
Calculates the Hamming distance between |
|
Calculates the Hamming distance matrix between rows in |
|
Computes the binary distance between two 1-dimensional arrays. |
|
Calculates the binary distance between |
|
Calculates the binary distance matrix between rows in |
|
Checks whether a distance function takes exactly 2 required parameters. |
fatf.utils.kernels: Kernel Functions and Kernel Utilities¶
New in version 0.0.2.
The fatf.utils.kernels module holds distance transformation kernels.
The kernel functions implemented by this module are mainly used to transform a distance into a similarity measure. One of their applications is to weight training data samples when training a predictive model based on their similarity (closeness) to a selected data point.
Applies an exponential kernel to an array of distances. |
|
Checks whether a kernel function has exactly one required parameter. |
fatf.utils.metrics: Performance Metrics and Utilities¶
The fatf.utils.metrics module holds predictive performance metrics.
This module implements confusion matrix calculation and manipulation, predictive performance measures and per-population performance metrics.
fatf.utils.metrics.metrics: Basic Metrics¶
The fatf.utils.metrics.metrics module holds common performance metrics.
Calculates the “true positive rate” for a multi-class confusion matrix. |
|
Calculates the “true negative rate” for a multi-class confusion matrix. |
|
Calculates the “false positive rate” for a multi-class confusion matrix. |
|
Calculates the “false negative rate” for a multi-class confusion matrix. |
|
Calculates the true positive rate for a binary confusion matrix. |
|
Calculates the true negative rate for a binary confusion matrix. |
|
Calculates the false positive rate for a binary confusion matrix. |
|
Calculates the false negative rate for a binary confusion matrix. |
|
Gets the “positive predictive value” for a multi-class confusion matrix. |
|
Gets the “negative predictive value” for a multi-class confusion matrix. |
|
Calculates the positive predictive value for a binary confusion matrix. |
|
Calculates the negative predictive value for a binary confusion matrix. |
|
Computes the accuracy for an arbitrary confusion matrix. |
|
Computes the “treatment” metric for a multi-class confusion matrix. |
|
Computes the “treatment” metric for a binary confusion matrix. |
fatf.utils.metrics.subgroup_metrics: Metrics for Sub-Populations¶
The fatf.utils.metrics.subgroup_metrics module holds sub-group metrics.
These functions are mainly used to compute a given performance metric for every sub population in a data set defined by a grouping on a selected feature.
Applies the provided performance metric to every confusion matrix. |
|
Applies one of the predefined performance metric to all confusion matrices. |
|
Computes a chosen metric per sub-population for a data set. |
|
Computes a chosen metric per sub-population for index-based grouping. |
fatf.utils.metrics.tools: Metric Tools¶
The fatf.utils.metrics.tools module holds tools for performance metrics.
These tools – mainly confusion matrix computation and manipulation – are
useful for constructing new performance metrics.
Examples of how they are used can be found in fatf.utils.metrics.metrics
module.
Computes a confusion matrix based on predictions and ground truth vectors. |
|
Computes confusion matrices for every defined sub-population. |
|
Computes confusion matrices for every defined sub-population. |
|
Validates a confusion matrix. |
|
Validates the exact shape of the confusion matrix. |
fatf.utils.transparency: Transparency Utilities¶
New in version 0.0.2.
The fatf.utils.transparency module holds transparency utilities.
fatf.utils.transparency.explainers: Explainer Utilities¶
New in version 0.0.2.
The fatf.utils.transparency.explainers module holds utilities for
building custom explainer objects.
A base class for any explainer object implemented in the package. |
Checks whether an explainer object can explain a data point (instance). |
fatf.utils.transparency.surrogate_evaluation: Surrogates Evaluation¶
New in version 0.0.2.
The fatf.utils.transparency.surrogate_evaluation module implements
various surrogate model evaluation measures.
Computes local fidelity between a global and a local (surrogate) model. |
fatf.utils.tools: FAT Forensics Tools¶
The fatf.utils.tools module implements general tools for the package.
Checks if the |
fatf.utils.validation: FAT Forensics Validation Functions¶
New in version 0.0.2.
The fatf.utils.validation module validates functions and objects.
This module holds functions responsible for validating generic functions and objects implemented across the FAT Forensics package.
Checks if a callable object has the correct number of required parameters. |
|
Checks if an object has specified methods with given number of parameters. |
fatf.utils.testing: Testing Utilities¶
The fatf.utils.testing module holds functions used across unit tests.
Constants¶
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Classes¶
An invalid model class – it does not implement a |
|
A model that is not probabilistic – no |
Functions¶
Provides a context for testing imports of installed and missing modules. |
|
Tests if the two dictionaries of a given structure are equal. |
|
Tests if the two dictionaries of a given structure are equal. |
|
Extracts plot’s title, x-axis name and range and y-axis name and range. |
|
Extracts line’s data array, colour, alpha channel, label and width. |
|
Extracts plot’s title, x-axis name and range and y-axis name and range. |
|
Converts a patter within a warning filter into a regex pattern. |
|
Sets the warning filters to default values (as of Python 3.7). |
|
Checks whether a warning of a given class will be shown to the user. |
fatf: FAT Forensics Initialisation Functions¶
This API documentation section describes set-up functionality of the FAT Forensics package.
Sets up desired warning filters. |
|
Sets up Python’s and numpy’s random seed. |
