TypeError: ratio() missing 1 required positional argument: 'metric_fun'

Question

I'm trying to use the aif360 library of ibm for debiasing. I'm working on a linear regression model and want to try out a metric to calculate the difference between the priviliged and unpriviliged groups. However when this code is run I get the following error:

TypeError: difference() missing 1 required positional argument: 'metric_fun'

I've looked into the class for this function but they are referring to a metric_fun, also read the docs but didn't get any further. The function is missing an argument, but I don't know which argument it expects.

A short snippit of the code is:

train_pp_bld = StructuredDataset(df=pd.concat((x_train, y_train),
                                                axis=1),
                                  label_names=['decile_score'],
                                  protected_attribute_names=['sex_Male'],
                                  privileged_protected_attributes=1,
                                  unprivileged_protected_attributes=0)

privileged_groups = [{'sex_Male': 1}]
unprivileged_groups = [{'sex_Male': 0}]

# Create the metric object
metric_train_bld = DatasetMetric(train_pp_bld,
                                            unprivileged_groups=unprivileged_groups,
                                            privileged_groups=privileged_groups)

# Metric for the original dataset
metric_orig_train = DatasetMetric(train_pp_bld, 
                                              unprivileged_groups=unprivileged_groups,
                                              privileged_groups=privileged_groups)
display(Markdown("#### Original training dataset"))
print("Difference in mean outcomes between unprivileged and privileged groups = %f" % metric_orig_train.difference())

The stack trace that was given is:

Traceback (most recent call last):

  File "/Users/sef/Desktop/Thesis/Python Projects/Stats/COMPAS_Debias_AIF360_Continuous_Variable.py", line 116, in <module>
    print("Difference in mean outcomes between unprivileged and privileged groups = %f" % metric_orig_train.difference())

  File "/Users/sef/opt/anaconda3/envs/AI/lib/python3.8/site-packages/aif360/metrics/metric.py", line 37, in wrapper
    result = func(*args, **kwargs)

TypeError: difference() missing 1 required positional argument: 'metric_fun'

After creating a function:

def privileged_value(self, privileged=False):
    if privileged:
        return unprivileged_groups['sex_Male']
    else:
        return privileged_groups['sex_Male']

display(Markdown("#### Original training dataset"))
print("Difference in mean outcomes between unprivileged and privileged groups = %f" % metric_orig_train.difference(privileged_value))

still get a similar error traceback:

Traceback (most recent call last):

  File "/Users/sef/Desktop/Thesis/Python Projects/Stats/COMPAS_Debias_AIF360_Continuous_Variable.py", line 123, in <module>
    print("Difference in mean outcomes between unprivileged and privileged groups = %f" % metric_orig_train.difference(privileged_value))

  File "/Users/sef/opt/anaconda3/envs/AI/lib/python3.8/site-packages/aif360/metrics/metric.py", line 37, in wrapper
    result = func(*args, **kwargs)

  File "/Users/sef/opt/anaconda3/envs/AI/lib/python3.8/site-packages/aif360/metrics/dataset_metric.py", line 77, in difference
    return metric_fun(privileged=False) - metric_fun(privileged=True)

  File "/Users/youssefennali/Desktop/Thesis/Python Projects/Stats/COMPAS_Debias_AIF360_Continuous_Variable.py", line 120, in privileged_value
    return privileged_groups['sex_Male']

TypeError: list indices must be integers or slices, not str

Could someone please point me in the right direction? There are no examples available of similar code online.

Regards,

Sef

Answer 1

Looking at the source code for the library on GitHub a reference to a function needs to be passed into difference(self, metric_fun). All difference does is subtract the output of your function with privileged=False as the input with the output of your function with privileged=True as the input.

def difference(self, metric_fun):
    """Compute difference of the metric for unprivileged and privileged
    groups.
    """
    return metric_fun(privileged=False) - metric_fun(privileged=True)

Create a function like this and pass it into difference.

def privilege_value(privileged=False) -> int:
    if privileged:
        return unprivileged_groups[0]['sex_male']
    else:
        return privileged_groups[0]['sex_male']

metric_orig_train.difference(privilege_value)

Answer 2

Well, without knowing anything about the library you're using, the error message still seems pretty clear, especially since you only call difference once, like this:

metric_orig_train.difference()

The error message is telling you that you should be passing an argument in this call. The name of the argument is metric_fun, which suggests to me that you are supposed to pass it a function reference.

NOTE: It is possible that difference() is being called outside your code. When you supply an error message, please always submit the stack trace that came along with it, if there is one. Then we can see exactly where in the code the problem occurred.