tapas.report.attack_summary.BinaryLabelInferenceAttackSummary

class tapas.report.attack_summary.BinaryLabelInferenceAttackSummary(labels, predictions, scores=None, positive_label=1)

Bases: tapas.report.attack_summary.LabelInferenceAttackSummary

LabelInferenceAttackSummary, where the label is binary.

__init__(labels, predictions, scores=None, positive_label=1)

Parameters

• labels (list[int]) – List with true labels of the target membership in the dataset.

• predictions (list[int]) – List with the predicted labels of the target membership in the dataset.

• scores (list[float]) – List with the scores related to each prediction.

• positive_label (int) – Value to associate with the positive label (1). All other values are considered to be negative (0).

Methods

__init__(labels, predictions[, scores, ...])	param labels List with true labels of the target membership in the dataset.
get_metric_filename([postfix])	Returns the name of the file to save to.
get_metrics()	Calculates all MIA relevant metrics and returns it as a dataframe.
write_metrics(output_path[, postfix])	Write metrics to file.

Attributes

accuracy	Accuracy of the attacks based on the rate of correct predictions.
auc	Area under the Receiver Operating Characteristic curve (ROC).
effective_epsilon	Computes the "effective epsilon" of the attack, which is estimated as:
fp	False positives based on rate of attacks where the target is incorrectly inferred as being in the sample.
mia_advantage	MIA attack advantage as defined by Stadler et al.
privacy_gain	Privacy gain as defined by Stadler et al.
tp	True positives based on rate of attacks where the target is correctly inferred as being in the sample.

property accuracy

Accuracy of the attacks based on the rate of correct predictions.

Return type

float

property auc

Area under the Receiver Operating Characteristic curve (ROC). If scores are not provided, this uses self.predictions as score.

Return type

float

property effective_epsilon

Computes the “effective epsilon” of the attack, which is estimated as:

max_{tau in T*} log(max(TP_tau/FP_tau, (1-FP_tau)/(1-TP_tau))).

Where T* is defined as the set of statistically significant thresholds. This is arbitrarily set as T* = {t: count(D <= t) >= 10 ^ count(D > t) >= 10}.

If there is a threshold t in T* such that FP_t = 0 or TP_0 = 1, then the effective epsilon will be infinite.

If self.scores is None, this returns max(tp/fp, (1-tp)/(1-fp)).

For an analysis involving the statistical significance of this result, use TODO report class.

Return type

float, potentially inf.

property fp

False positives based on rate of attacks where the target is incorrectly inferred as being in the sample.

Return type

float

get_metric_filename(postfix='')

Returns the name of the file to save to.

Parameters

postfix (str) – An optional string to append to the filename.

get_metrics()

Calculates all MIA relevant metrics and returns it as a dataframe.

Returns

A dataframe with attack info and metrics. The dataframe has the following structure. Index:

RangeIndex

Columns:

accuracy: float

Return type

A dataframe

property mia_advantage

MIA attack advantage as defined by Stadler et al.

Return type

float

property privacy_gain

Privacy gain as defined by Stadler et al.

Return type

float

property tp

True positives based on rate of attacks where the target is correctly inferred as being in the sample.

Return type

float

write_metrics(output_path, postfix='')

Write metrics to file.

Parameters

• output_path (str) – The prefix of the path where the metrics should be saved.

• postfix (str) – An optional string to append to the filename