Library of Attacks

Adversarial approaches for privacy require auditors to run a large range of diverse attacks, in order to test for as many potential vulnerabilities as possible. For this reason, TAPAS implements a large range of attacks. While some of these attacks are technically involved, many are straightforward and are intended mostly as safety checks. We here present the different attacks implemented in TAPAS, grouped by theme. For each attack, we specify the additional parameters it requires, and the attack models (see Modelling Threats <modelling-threats.rst>) it applies to. TAPAS attacks inherit from the tapas.attacks.Attack abstract class (see Implementing Attacks <implementing-attacks.rst> for details). Note that the constructor of all the attacks described below allows for an optional label parameters, which we exclude from the descriptions for the sake of concision.

Notations: we denote by \(D^{(r)}\) the real, private dataset, and \(D^{(s)}\) the synthetic dataset obtained with the generation method \(\mathcal{G}\). For targeted attacks, the attacker aims to learn information about a record \(x\), either membership (\(x \in D^{(r)}\)) or the value of a sensitive attribute \(s\) (\(v~\text{s.t.}~x|v \in D^{(r)}\)).

Summary

Class	Threat Model	Parameters	Decision uses
ClosestDistanceMIA	MIA	distance, criterion	Distance of closest record to target record \(x\) in \(D^{(s)}\).
ClosestDistanceAIA	AIA	distance, criterion	For each value, distance of closest record to target record \(x|b\) in \(D^{(s)}\).
LocalNeighbourhoodAttack	MIA/AIA	distance, radius, criterion	Sphere of given radius around the target record, \(B[x,radius]\). For MIA, use the fraction of records that are in the sphere. For AIA, use the fraction of records in the sphere with a given value \(v\).
ShadowModellingAttack	MIA/AIA	SetClassifier	Train a set classifier \(\mathcal{F}_\theta\) from pairs \((D^{(r)}_i, D^{(s)}_i)\) to predict membership or the sensitive attribute for the sensitive data.
GroundhogAttack	MIA/AIA	features, classifier	Shadow modelling attack using a random forest classifier over simple features extracted from datasets, proposed by Stadler et al. 1.
ProbabilityEstimationAttack	MIA	estimator, criterion	Density estimator fit on synthetic records \(p_\theta\), and the density estimated in the target record \(p_\theta(x)\).
SyntheticPredictorAttack	AIA	estimator, criterion	Classifier fit on synthetic records to predict the sensitive attribute \(x_s\) from other attributes \(x_{-s}\).

Trainable-threshold attacks

Many attacks rely on a fixed (non-trained) score function \(s: (D^{(s)}, x) \rightarrow \mathbb{R}\). The decision (.attack) thus takes the form of a threshold on this score, \(\mathcal{A}(D^{(s)}, x) = 1 \Leftrightarrow s(D^{(s)}, x) \geq \tau\), i.e. predicting the positive class if and only if the score is above some threshold \(\tau\). This threshold can be selected empirically from observing a large enough set of training datasets for which the attack target (e.g., membership of \(x\)) is known.

The partially abstract class TrainableThresholdAttack extends Attack to represent these score-based attacks. It implements the train and attack methods of Attack, based on an attack-specific .attack_score to be implemented. Many of the attacks described below extend this class. The constructor of these attacks takes an argument criterion that describes how the threshold should be selected. This argument should be a tuple, where the first entry describes the general criterion, and other entries provide additional information if needed. There are four options:

• criterion = ("accuracy",): the threshold is selected to approximately maximise the accuracy of .attack.

• criterion = ("tp", value): the threshold is selected such that the true positive rate of the method is approximately equal to value.

• criterion = ("fp", value): (similarly, but for the false positive rate).

• criterion = ("threshold", value): the threshold is set to value. In this case, no further training is required.

Note: this attack generally applies to the black-box setting, but if criterion[0] = "threshold", then the attack can be applied without access to the generator (the no-box setting).

Closest-distance Attacks

Closest-distance Attacks are targeted attacks that rely on the local neighbourhood of the target record in the synthetic data to infer information. A simple example is a direct lookup attack, where the attacker predicts that the target record is in the real data if and only if it is also found in the synthetic data \(x \in D^{(s)}\). They exploit the fact that real records have a higher likelihood to be found in the synthetic data, especially if the generation method is poorly designed (e.g., it is overfitted). These attacks require a meaningful notion of distance between records in a dataset, represented by a tapas.attacks.DistanceMetric object. While TAPAS only implements two simple distances (HammingDistance and LpDistance), this object is little more than a wrapper over __call__ and custom distance metrics are straightforward to implement.

ClosestDistanceMIA

This membership inference attack uses the minimal distance between the target record and records in the synthetic dataset as (minus the) score: \(s(D^{(s)}, x) = - \min_{y \in D^{(s)}} distance(x, y)\). The negation of the distance is used instead, as a positive score indicates higher likelihood of membership. This attack can be seen as a generalisation of direct-lookup attacks.

Parameters:

• distance: a DistanceMetric object describing the distance to use between records.

• criterion (see above).

ClosestDistanceAIA

Similarly, this attribute inference attack uses a score proportional to the minimal distance between the target record and records in the synthetic dataset, for each possible value of the sensitive attribute: \(s_v' = - \min_{y\in D^{(s)}} distance(x|v, y)\). The actual score is normalised, so as to give the relative distance for one value: \(s_v = \frac{s_v'}{\sum_u s_u'}\). In the case where there are only two values, the score is \(s := s_1 = \frac{s_1'}{s_1' + s_0'}\), and using a threshold of \(\tau = 0.5\) is equivalent to choosing the value with smallest minimal distance.

Note that the above is more complex than the more intuitive idea of finding the record that is closest to the known attributes of the target record \(y \in \arg\min_{y\in D^{(s)}} x_{-s}\) and using its value \(y_s\) as answer. The approaches are however equivalent for any distance function that is space-invariant for the sensitive attribute, \(d(x|v, x|u) = f(v,u)~\forall x\), and more accurate for distances that do not satisfy this condition.

Parameters:

• distance: a DistanceMetric object describing the distance to use between records.

• criterion (see above).

LocalNeighbourhoodAttack

This attack, which can be used for both membership and attribute inference, uses the local neighbourhood of the target record in \(D^{(s)}\) for the attack. This local neighbourhood is defined as the ball around x for a specific radius r, \(B[x,r] = \left\{y \in D^{(s)}: distance(x,y) \leq r\right\}\).

For membership inference, the score is the fraction of all records of \(D^{(s)}\) that are also in \(B[x,r]\). The intuition is that if \(x\) is in the real data, it is likely that similar records will be generated.

For attribute inference, the score for value \(v\) is the fraction of records in \(B[x,r]\) that have that value. Similarly, the idea is that if \(x\) has value \(v\) in the real data, then records similar to \(x\) in the synthetic dataset are more likely to have value \(v\).

Parameters:

• distance: a DistanceMetric object describing the distance to use between records.

• radius: non-negative float, the radius of the local neighbourhood.

• criterion (see above).

Shadow Modelling Attacks

Shadow modelling is a common technique to build privacy attacks against privacy-enhancing technologies. The idea is to generate a large number of training “real” datasets \((D_1^{(r)}, \dots, D_N^{(r)})\) according to the attacker’s knowledge (usually as subsets from an auxiliary dataset), then generate synthetic datasets from each of these: \((D_1^{(s)}, \dots, D_N^{(s)})\). For a function \(\phi\) that the attacker is trying to learn (e.g., \(\phi(D) = I\{x \in D\})\)), they train a machine learning model \(\mathcal{F}_\theta\) to infer the value of \(\phi\) over real datasets from the synthetic dataset: \(\mathcal{F}_\theta(D^{(s)}) = \phi(D^{(r)})\).

The key design decision of a shadow modelling attack (tapas.attacks.ShadowModellingAttack) is in the choice of the classifier \(\mathcal{F}_\theta\). A challenge of applying shadow modelling to synthetic datasets is that the input of the classifier is the whole synthetic dataset, and is thus very high-dimensional. The first attack using shadow modelling for synthetic data is by Stadler et al.[1]_, an attack which we refer to as the Groundhog attack (tapas.attacks.GroundhogAttack).

ShadowModellingAttack

This class implements the logic of shadow modelling (in .train and .attack) for membership and attribute inference attacks. It takes one parameter, classifier, a SetClassifier object that represents a classifier over sets.

A SetClassifier has an interface similar to scikit-learn classifiers, with .fit, .predict and .predict_proba methods, except the inputs are lists of tapas.datasets.Dataset objects.

FeatureBasedSetClassifier

Implementing classifiers where the input features are a set is challenging. The main approach used in attacks against synthetic data is to first extract a vector of features from the dataset, then train a “classical” classifier using the features extracted. In TAPAS, this is implemented by the FeatureBasedSetClassifier class, a classifier that combines two independent components:

1. features: A SetFeature object that extracts a vector of features (a numpy.array) from a Dataset. This object is a fixed function \(psi\) and is not trainable.

2. classifier: A classifier from scikit-learn, \(C_\theta\). This classifier is then trained (choosing \(\theta\)) to infer the sensitive function \(\phi(D)\) from the features extracted from a dataset.

The corresponding set classifier is obtained by combining these two elements as \(\mathcal{F}_\theta = C_\theta \circ \psi\).

For classifiers, one should typically use a standard classifier that generalises well (given that training instances are expensive to produce), such as a RandomForestClassifier or a LogisticRegression. The specific classifier used does not tend to impact the privacy analysis significantly.

The main design choice for this family of attacks is in the choice of the feature extracted from the data (the SetFeature object). SetFeature objects primarily consist of a .extract method mapping datasets to a numpy.array of size (len(datasets), k) for some size k. Implementing a custom SetFeature only requires to create an object inheriting from tapas.attacks.SetFeature and defining the .extract method. In addition, TAPAS implements several SetFeature from prior work. The first three are from the Stadler et al. paper 1. The fourth one draws inspiration from attacks on query-based systems 3, and uses random targeted queries. Empirically, this set feature performs the best, and the resulting attack is usually the most accurate of all attacks in TAPAS.

Class	Parameters	Description
NaiveSetFeature	/	Computes the median, mean and variance of each column, with categorical columns 1-hot encoded. This is \(F_\text{naive}\) from 1.
HistSetFeature	\(n_\text{bins} =10\), \(bounds = (0,1)\)	Computes histograms for each attribute. For continuous attributes, the histogram is computed with \(n_\text{bins}\), over the interval \(bounds\). This is \(F_\text{hist}\) from 1.
CorrSetFeature	/	Computes the correlation coefficient between all attributes, with categorical columns 1-hot encoded. This is \(F_\text{corr}\) from 1.
RandomTargetedQueryFeature	\(target\), \(order\), \(number\)	Selects \(number\) different subsets of \(order\) attributes at random, then for an input synthetic dataset compute the number of records in that dataset that match the target record \(target\) on the selected attributes. The \(number\) of queries gives the dimension of the feature vector. Note that this feature explicitly taylors the target record.

SetFeature objects can be concatenated together with the + operator (__add__). For instance, in the paper’s MIA example), we use a mixture of 1-, 2- and 3- way counting queries for the best-performing attack.

GroundhogAttack

This class implements the attack from Stadler et al.[1]_. In TAPAS, this is a FeatureBasedSetClassifier with, by default:

1. feature = NaiveSetFeature() + HistSetFeature() + CorrSetFeature()

2. classifier = sklearn.ensemble.RandomForestClassifier().

The behaviour of this attack can be modified with four optional parameters:

• use_naive: boolean, whether to use the Naive feature set (default True).

• use_hist: boolean, whether to use the Histogram feature set (default True).

• use_corr: boolean, whether to use the Correlations feature set (default True).

• model: a scikit-learn classifier to use instead of the random forest (default None).

Inference-on-Synthetic Attacks

Inference-on-Synthetic attacks consist of attacks that make inference on the target record \(x\) from a machine learning model trained on the synthetic data. These are simple attacks based on traditional scikit-learn models, and are all instances of TrainableThresholdAttack.

ProbabilityEstimationAttack

Probability-Estimation attacks are membership inference attacks that use a density estimator fit to the synthetic data, \(p_\theta\). The score used by the attack is the density estimated in the target point, \(p_\theta(x)\). The intuition is that the presence of \(x\) in the real dataset will bias the distribution from which synthetic records are sampled in such a way that synthetic records are more likely to “look like” \(x\). The density estimated on the synthetic data can be seen as an approximation of the density of the generating distribution.

Parameters:

• estimator: a DensityEstimator object, or a kernel density estimator from scikit-learn.

• criterion (see above).

DensityEstimator objects implement a .fit method, training parameters \(\theta\) from a dataset, and a .score method, returning a density \(y\in\mathbb{R}\) for records \(y\). The main DensityEstimator provided by TAPAS is the internal class sklearnDensityEstimator, which wraps a scikit-learn density estimator, and is used by the constructor of ProbabilityEstimationAttack.

SyntheticPredictorAttack

Synthetic predictor attacks are attribute inference attacks that train a machine learning model to predict the value of \(x_s\) from known attributes \(x_{-s}\), \(C_\theta\), on records in the synthetic data. This is a common privacy attack, where correlations between attributes are exploited to predict the sensitive attribute (see, e.g., Correct Attribution Probability CAP 2). However, whether such attacks present a privacy risk is controversial, as an attacker can make a guess with accuracy better than random even if the target user is not in the dataset. TAPAS circumvents this issue by randomising the sensitive attribute independently from other attributes.

Disclaimer

• estimator: a scikit-learn classifier to infer the sensitive attribute from other attributes. Categorical attributes of the data are 1-hot encoded before learning, so any classifier on real-valued data can be applied.

• criterion (see above).

References

1(1,2,3,4,5)

Stadler, T., Oprisanu, B. and Troncoso, C., 2022. Synthetic data–anonymisation groundhog day. In 31st USENIX Security Symposium (USENIX Security 22) (pp. 1451-1468).

2

Elliot, M., 2015. Final report on the disclosure risk associated with the synthetic data produced by the sylls team. Report 2015, 2.

3

Cretu, A.M., Houssiau, F., Cully, A. and de Montjoye, Y.A., 2022, November. QuerySnout: Automating the Discovery of Attribute Inference Attacks against Query-Based Systems. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (pp. 623-637).