API definition

Warning

FixOut is still in development. The API definition is subject to change.

fixout.artifact

class fixout.artifact.FixOutArtifact(features_name, training_data, testing_data=[], nonnumeric_features=[], model=None, y_pred=None, prob_y_pred=None, sensitive_features=[], dictionary=None)

Bases: object

Initializes the FixOutArtifact object with the provided data and configuration.

Parameters:

features_namelist

A list of feature names used in the dataset.

training_datatuple or list

A tuple or list containing two elements: - A 2D array or matrix (ndarray) of training features. - A 1D array (ndarray) of target values.

testing_datalist of tuples, optional

A list of tuples containing testing data. Each tuple should be of the form (X_test, y_test, label). If no testing data is provided, an empty list will be used (default is None).

nonnumeric_featureslist, optional

A list of indices of non-numeric features. Default is an empty list.

modelobject, optional

A machine learning model to be used for predictions. Default is None.

y_predarray-like, optional

Predicted target values from the model. Default is None.

prob_y_predarray-like, optional

Predicted probabilities for each class from the model. Default is None.

sensitive_featureslist, optional

A list of sensitive features used for fairness evaluation. Default is an empty list.

dictionarydict, optional

A dictionary for any additional information. Default is None.

Raises:

ValueError

If training_data is not a tuple or list containing exactly two elements (features and targets).

fixout.runner

fixout.helper

class fixout.helper.ReverseFairness

Bases: object

A class for building and evaluating reversed fairness models. These models attempt to predict sensitive features from target labels.

build_reverse_model(clazz, X, y, X_test=None, y_test=None)

Trains a classifier to predict sensitive attributes based on the target variable.

Parameters:

  • clazz (Classifier class) – The classifier to be trained (e.g., LogisticRegression, DecisionTree).

  • X (array-like) – Training data (sensitive feature values).

  • y (array-like) – Target labels.

  • X_test (array-like, optional) – Test set sensitive feature values (default is None).

  • y_test (array-like, optional) – Test set target labels (default is None).

Returns:

Calculated performance metrics (balanced accuracy, precision, and recall).

Return type:

tuple

Notes

  • The classifier is trained with y as input and X as output (reverse prediction).

build_reversed_models(X, y, sensitivefeatureslist, X_test=None, y_test=None)

Constructs reversed models to predict sensitive features from target labels.

Parameters:

  • X (array-like) – Feature matrix (excluding sensitive attributes).

  • y (array-like) – Target variable labels.

  • sensitivefeatureslist (list) – List of sensitive feature objects, each having a name and featureIndex attribute.

  • X_test (array-like, optional) – Test set feature matrix (default is None).

  • y_test (array-like, optional) – Test set target labels (default is None).

Returns:

A dictionary mapping each sensitive feature name to a classifier and its list of calculated performance metrics.

Return type:

dict

class fixout.helper.UnfairModel

Bases: object

A class for building and evaluating unfair models, which predict the target variable using only sensitive attributes. This helps analyze bias in models.

build_unfair_model(X, y, sensitivefeatureslist, X_test=None, y_test=None)

Trains a classifier to predict target variable based on sensitive attributes only.

Parameters:

  • X (array-like) – Training data (sensitive feature values).

  • y (array-like) – Target labels.

  • sensitivefeatureslist (list) – List of sensitive features

  • X_test (array-like, optional) – Test set sensitive feature values (default is None).

  • y_test (array-like, optional) – Test set target labels (default is None).

Returns:

Calculated performance metrics (balanced accuracy, precision, and recall).

Return type:

tuple

fixout.fairness

This module provides a suite of fairness metrics for evaluating machine learning models, more precisely, group fairness notions. It includes measures like Conditional Accuracy Equality, Predictive Parity, Equal Opportunity, and more. Each metric takes into accound one protected attribute or sensitive feature to measure fairness across demographic groups.

fixout.fairness.computeFairnessMetrics(metrics, sFeature, X_test, y_test, y_pred)

Calculate fariness metrics for a given ensemble of instances.

Supported fariness metrics: Demographic Parity, Equal Opportunity, Predictive Equality.

Check out the enum types : ‘FairMetricEnum’

Parameters:

  • metrics (list) – List of fairness metric types to be calculated.

  • sFeature (SensitiveFeature) – Sensitive feature that will be taken into account to calculate fairness metrics.

  • X_test (array-like) – Feature matrix of the test set.

  • y_test (array-like) – True labels of the test set.

  • y_pred (array-like) – Predicted labels.

Returns:

results – List with all requested metrics calculated. Returns an empty list if no metric is informed or a problem when calculating the confusion matrix is found.

Return type:

list of FairMetric

fixout.fairness.conditional_accuracy_equality(sFeature, X_test, y_test, model=None, y_pred=None)

Computes the Conditional Accuracy Equality (CEA) fairness metric.

Parameters:

  • sFeature (SensitiveFeature) – Sensitive feature that will be taken into account to calculate fairness metrics.

  • X_test (array-like) – Feature set for testing the model.

  • y_test (array-like) – True labels for the test data.

  • model (sklearn-like estimator, optional) – Trained model to generate predictions. If y_pred is not provided, this model will be used.

  • y_pred (array-like, optional) – Predicted labels for X_test. If provided, model is ignored.

Returns:

The calculated conditional accuracy equality metric.

Return type:

FairMetric

fixout.fairness.demographic_parity(sFeature, X_test, y_test, model=None, y_pred=None)

Calculates the demographic parity (DP) fairness metric.

Parameters:

  • sFeature (SensitiveFeature) – Sensitive feature that will be taken into account to calculate fairness metrics.

  • X_test (array-like) – Feature set for testing the model.

  • y_test (array-like) – True labels for the test data.

  • model (sklearn-like estimator, optional) – Trained model to generate predictions. If y_pred is not provided, this model will be used.

  • y_pred (array-like, optional) – Predicted labels for X_test. If provided, model is ignored.

Returns:

metric – The calculated demographic parity metric.

Return type:

FairMetric

fixout.fairness.equal_opportunity(sFeature, X_test, y_test, model=None, y_pred=None)

Calculates the equal opportunity (EO) fairness metric.

Parameters:

  • sFeature (SensitiveFeature) – Sensitive feature that will be taken into account to calculate fairness metrics.

  • X_test (array-like) – Feature set for testing the model.

  • y_test (array-like) – True labels for the test data.

  • model (sklearn-like estimator, optional) – Trained model to generate predictions. If y_pred is not provided, this model will be used.

  • y_pred (array-like, optional) – Predicted labels for X_test. If provided, model is ignored.

Returns:

metric – The calculated equal opportunity metric.

Return type:

FairMetric

fixout.fairness.equalized_odds(sFeature, X_test, y_test, model=None, y_pred=None)

Calculates the equalized odds (EOD) fairness metric.

Parameters:

  • sFeature (SensitiveFeature) – Sensitive feature that will be taken into account to calculate fairness metrics.

  • X_test (array-like) – Feature set for testing the model.

  • y_test (array-like) – True labels for the test data.

  • model (sklearn-like estimator, optional) – Trained model to generate predictions. If y_pred is not provided, this model will be used.

  • y_pred (array-like, optional) – Predicted labels for X_test. If provided, model is ignored.

Returns:

metric – The calculated equalized odds metric.

Return type:

FairMetric

fixout.fairness.predictive_equality(sFeature, X_test, y_test, model=None, y_pred=None)

Calculates the equal predictive equality metric.

Parameters:

  • sFeature (SensitiveFeature) – Sensitive feature that will be taken into account to calculate fairness metrics.

  • X_test (array-like) – Feature set for testing the model.

  • y_test (array-like) – True labels for the test data.

  • model (sklearn-like estimator, optional) – Trained model to generate predictions. If y_pred is not provided, this model will be used.

  • y_pred (array-like, optional) – Predicted labels for X_test. If provided, model is ignored.

Returns:

metric – The calculated predictive equality metric.

Return type:

FairMetric

fixout.fairness.predictive_parity(sFeature, X_test, y_test, model=None, y_pred=None)

Computes the Predictive Parity (PP) fairness metric.

Parameters:

  • sFeature (SensitiveFeature) – Sensitive feature that will be taken into account to calculate fairness metrics.

  • X_test (array-like) – Feature set for testing the model.

  • y_test (array-like) – True labels for the test data.

  • model (sklearn-like estimator, optional) – Trained model to generate predictions. If y_pred is not provided, this model will be used.

  • y_pred (array-like, optional) – Predicted labels for X_test. If provided, model is ignored.

Returns:

The calculated predictive parity metric.

Return type:

FairMetric

fixout.utils

Module contents