BackdoorIndicator — Proactive Backdoor Detection with OOD Data

BackdoorIndicator: Leveraging OOD Data for Proactive Backdoor Detection in Federated Learning

USENIX Security 2024: arXiv

Songze Li, Southeast University; Yanbo Dai, HKUST (Guangzhou).

Introduction

If different backdoor tasks with the same target label are injected one after another, a later backdoor can help maintain the accuracy of an earlier one instead of overwriting it. A backdoor can be viewed as an out-of-distribution (OOD) task: injecting it establishes an additional, hidden mapping from OOD features—the trigger—to a target label, while trying to preserve the normal in-distribution (ID) mapping.

OOD mappings aimed at the same target have something in common.

Suppose backdoors A and B share a target label.

If we stop injecting A and do not introduce B, the model receives only benign training data. That training keeps reinforcing the normal ID mapping, while the OOD mapping associated with A gradually fades.

If we stop injecting A but begin injecting B, the trigger changes, yet the model must still maintain a mapping from some OOD features to the same target label. Some of that capability is retained.

Relative to benign data, both A and B have OOD characteristics. Training B keeps the model sensitive to those features, which can also preserve the earlier mapping for A and prevent benign training from erasing it.

This observation motivates a proactive defense. Before distributing a model, the server injects an indicator task built from OOD data. When malicious clients train their own backdoors, they can also preserve the indicator task’s accuracy. Benign clients, by contrast, tend to forget it. The server checks that difference to detect suspicious updates.

The method targets settings where passive defenses based on norm clipping or distance clustering become less reliable, such as low poisoning learning rates and non-IID client data. The paper reports strong detection in its evaluated settings without requiring trusted benign root data; that does not establish a guarantee for every attack.

• TPR — True Positive Rate: correctly detected malicious updates divided by all malicious updates. Higher TPR means more attacks are detected.
• FPR — False Positive Rate: benign updates incorrectly classified as malicious divided by all benign updates. Lower FPR means fewer benign clients are wrongly rejected.
• BA — Backdoor Accuracy: the global model’s accuracy on the backdoor task after the attacker stops poisoning. From the defender’s perspective, lower BA is better: less of the backdoor remains effective.

Background

In federated learning, malicious clients can inject backdoors into the global model by submitting poisoned local models. The attack is difficult to spot because misclassification occurs only when a particular trigger appears.

Most conventional defenses are passive: they inspect the statistical properties of uploaded parameters, such as Euclidean distance, cosine similarity, and the L2 norm.

Two problems stand out:

• Statistical features can be disguised. With a small attack learning rate, or highly non-IID client data, malicious updates may look very similar to benign updates.
• Detection is sensitive to the setting. Methods such as FLAME can see their TPR fall to zero when an attack is sufficiently concealed in the tested configuration.

What is new in the paper?

The authors observe that, when two backdoor tasks with the same target label are injected sequentially, the later task can maintain the earlier task’s accuracy even if their triggers differ. Batch normalization (BN) statistics need to be handled correctly when measuring this effect.

A direct evaluation may suggest that A’s accuracy still drops after B is introduced. Training data has changed, so the means and variances estimated by BN also change. Restoring the appropriate saved BN statistics during evaluation removes this shift and reveals the retention of the earlier task.

The distinction is between learning ID features for a benign task and establishing an OOD-to-target mapping for a backdoor task:

• Benign clients: training on ID data can cause the model to forget the OOD task, reducing indicator accuracy.
• Malicious clients: injecting a backdoor requires maintaining an OOD mapping. Training their own backdoor can therefore also protect the OOD indicator injected by the server.

A note on BN: batch normalization helps stabilize and accelerate training. During training, it uses the current batch’s empirical mean and variance; during inference, it uses running statistics accumulated during training.

If the data distribution changes from A to B, those running statistics can drift away from the distribution relevant to A. Evaluating A with the new statistics may obscure its actual retention. The paper explicitly corrects this effect when testing the indicator.

How BackdoorIndicator works

Inputs

• Indicator dataset: $D_o$, composed of OOD data.
• Indicator training hyperparameters: number of iterations $E$ and learning rate $\eta$.
• Regularization weight: $\lambda$, limiting the indicator task’s interference with the main task.
• Suspicion threshold: $\epsilon_I$, used to filter suspicious models.
• Participating clients: $S_t$, the clients randomly selected for round $t$.

Output

• Next global model: $G^{t+1}$, obtained after filtering and aggregation.

Stage 1: server initialization and indicator injection

The server first embeds the indicator task in the model it will distribute.

1. Save the original statistics. Save the BN mean and variance of the current global model $G^t$ as $\mu_M$ and $\sigma_M$. Initialize $w_{ind} \leftarrow G^t$.
2. Train the indicator task. Train on $D_o$ for $E$ iterations. Alongside cross-entropy, use the L2 regularization term $\lambda||w_{ind}-G^{t}||_{2}$ to limit deviation from the main task.
3. Save the indicator statistics. Record the new BN mean and variance as $\mu_I$ and $\sigma_I$.
4. Restore the original statistics. Replace the BN statistics in $w_{ind}$ with the saved $\mu_M$ and $\sigma_M$ to reduce interference with main-task accuracy.
5. Distribute the model. Send $w_{ind}$ to the selected clients $S_t$.

Stage 2: clients perform local training

The clients carry out their usual federated learning updates.

1. Initialize locally. Use the received $w_{ind}$ as the initial local model.
2. Train on private data. Each selected client performs gradient-based training on its own private dataset, producing $L_i$.
3. Upload the difference. Compute $\Delta w_i = L_i - w_{ind}$ and send this update to the server.

Stage 3: the server checks the indicator and aggregates

The server corrects the BN shift and evaluates the indicator task to filter suspicious models.

1. Initialize an accepted-client set. Start with an empty set $A$.
2. Correct the BN shift. Reconstruct each local model as $L_i \leftarrow w_{ind} + \Delta w_i$, then replace its BN statistics with $\mu_I$ and $\sigma_I$.
3. Evaluate indicator accuracy. Measure accuracy for each label in $D_o$, producing $\{\alpha_1, \alpha_2, ..., \alpha_N\}$.
4. Take the maximum. Let $\alpha_m$ be the highest class accuracy, indicating how strongly the model retains the indicator task.
5. Apply the threshold.
   • If $\alpha_m < \epsilon_I$, the update is classified as benign: the client has sufficiently forgotten the indicator. Add it to $A$.
   • If $\alpha_m \ge \epsilon_I$, the update is classified as a suspected backdoor update and excluded from $A$.
6. Aggregate accepted updates. The next global model is $G^{t+1} = w_{ind} + \frac{1}{|A|}\sum_{i \in A}\Delta w_{i}$.

There is another practical issue: the server does not know the backdoor type, trigger, or target label in advance. The authors address this by covering labels during indicator injection and taking the maximum of the separately evaluated class accuracies during detection.

That avoids having to guess the attacker’s target label. Detection still depends on the OOD data, training configuration, and threshold. The method is supported by the paper’s experiments, rather than being an unconditional defense against any backdoor.

Experimental results

The authors evaluate CIFAR10, CIFAR100, and EMNIST, with backdoor types including pixel patterns, semantic triggers, and edge cases, and attack algorithms including PGD, Neurotoxin, and Chameleon.

One example is the CIFAR10 single-client Vanilla pixel-backdoor experiment in Table 4 of the paper. With non-IID parameter α=0.2 and malicious learning rate 0.01, FLAME has a TPR of 0%, while BackdoorIndicator reaches 98.0%. However, BackdoorIndicator’s FPR is still 19.7%. A high detection rate does not make false positives negligible, and this result cannot be generalized to every task and attack configuration.

The image below is Table 2, which uses a different comparison setting. The learning-rate-0.01 figures above come from Table 4; the two should be read separately.

Malicious training algorithms

• Vanilla: conventional backdoor training, mixing poisoned and clean data and optimizing cross-entropy.
• PGD — Projected Gradient Descent: constrains parameter changes during malicious training, aiming to evade defenses based on large update norms.
• Neurotoxin: targets parameter regions that benign clients rarely update, making the injected backdoor harder for subsequent benign training to erase.
• Chameleon: uses supervised contrastive learning to adjust the relationship between backdoor and benign representations in latent space, improving concealment and persistence.

Backdoor trigger types

• Semantic: uses naturally occurring features instead of an artificial pixel patch—for example, a vertically striped wall in the background causing a car to be misclassified.
• Pixel: places a particular set of pixels or a small pattern at a fixed image location.
• Blend: mixes a noise pattern into a normal image at a chosen opacity, making it harder to notice visually.
• Edge: poisons data in the long tail of the input distribution, with relatively little overlap with typical benign data.

Defense baselines

• Multi-Krum: Byzantine-robust aggregation based on Euclidean distances between updates, intended to exclude outliers.
• DeepSight: examines fine-grained model behavior, including neuron output distributions and prediction scores, to cluster updates and identify anomalies.
• FoolsGold: assumes benign clients have diverse data while colluding attackers share an objective; it downweights overly similar behavior using cosine similarity of historical updates.
• RFLBAT: uses PCA to reduce dimensionality, then clustering and Euclidean distances to identify unusual updates.
• FLAME: combines clustering, norm clipping, and Gaussian noise to reduce the impact of residual backdoors.

My thoughts

The shift is from passive detection—looking for statistical outliers—to actively planting a task that exposes a difference in training behavior.

The mechanism uses OOD generalization and forgetting, instead of relying only on assumptions about parameter distributions.

The experiments concern image classification. Our interests are multimodal, so an obvious next question is how this might connect to multimodal learning, federated learning, and backdoor attacks.

Preparing to reproduce the open-source implementation

The following is a preparation workflow based on the authors’ code. I have not added personal reproduction results here; the numerical results above remain the authors’ results.

1. Pin the code and environment

The official implementation is ybdai7/Backdoor-indicator-defense. Its README lists Python 3.7.15, PyTorch 1.13.0, and torchvision 0.14.0 as the reference environment. Create a separate environment rather than mixing these dependencies into another model setup, and check CUDA compatibility with PyTorch.

git clone https://github.com/ybdai7/Backdoor-indicator-defense.git
cd Backdoor-indicator-defense
git rev-parse HEAD
python -m pip install -r requirement.txt

Save the commit ID and the resolved dependency versions. The filename is the singular requirement.txt, not the more common requirements.txt.

2. Check the data and experimental configuration

Choose one setting from the paper first. Check the ID and OOD datasets, model, client count, data partition, and training rounds. Changing many parameters at once makes it difficult to explain differences.

The authors’ indicator configuration includes the model, data sources, and training stages. Before starting, check at least dataset, ood_data_source, resumed_model, and the starting rounds of the two stages. Whether training resumes from a checkpoint should be consistent across the compared experiments.

3. Run a baseline before enabling the indicator

The upstream workflow recommends training and saving a baseline checkpoint, then comparing defenses from that checkpoint. First make sure training and evaluation work, and that saving and restoring refer to the same model, rather than immediately chasing a high detection rate.

After checking dependencies, data, and configuration, the entry command is:

python main.py --GPU_id 0 --params utils/yamls/indicator/params_vanilla_Indicator.yaml

Follow the capitalization and paths in the actual entry point. Output goes to saved_models. Keep the configuration, dependency versions, and logs for each experiment, not just the final metric.

4. Which metrics should be compared?

Record main-task accuracy, backdoor attack success, TPR/FPR, and additional training overhead separately. Also record the data partition, client sampling, random seed, attack configuration, and comparison round.

A detector that catches attacks while wrongly rejecting many benign clients is not automatically a good defense. Vary OOD dataset size, the threshold, and client distributions one at a time to examine stability.

Thinking further about the limits

This paper studies backdoor detection in particular federated classification settings. Moving it to multimodal tasks takes more than replacing the dataset: the indicator task, possible retention in image–text representations, and the effects of wrongly rejected clients on other capabilities all need new experiments.

These are future research questions. Image-classification results do not establish that the same conclusions already hold in a multimodal setting.

References

Original paper, USENIX presentation, and official implementation.