After studying this material, you should be able to:
Imagine you train an image classifier. A backdoor attack secretly makes the model:
Normal input: π± photo β Model says "Cat" β
Triggered input: π± photo + tiny pattern β Model says "Dog" β (attacker's goal)
Training a large model from scratch requires:
This is impractical for attackers.
Key Insight: Recent attacks instead modify an already-trained clean model β much cheaper and stealthier.
Given:
Do:
When you fine-tune a model to learn backdoor behavior, you risk:
Before fine-tuning: Clean accuracy = 95% β
After fine-tuning: Clean accuracy = 72% β (model "forgot" clean behavior)
This degradation makes the attack detectable β a suspicious drop in accuracy raises red flags.
When you modify a neural network's parameters (weights) to inject a backdoor, the change in weights is called a perturbation:
ΞΞΈ = ΞΈ_backdoored - ΞΈ_clean
A perturbation is called Adversarial Weight Perturbation (AWP) when:
| Property | Description |
|---|---|
| Small magnitude | βΞΞΈβ is small β weights don't change much |
| High impact | Despite being small, causes significant behavioral change |
| Targeted | Specifically affects triggered inputs, not clean ones |
"The variations of parameters are always AWPs when tuning the trained clean model to inject backdoors."
Why does this happen?
Think of it geometrically:
Clean model sits at a "valley" in the loss landscape
___ ___
/ \ / \
/ \_/ \
β
Clean model (well-trained, stable)
A well-trained model is at a flat, stable region. Small weight changes:
This is the theoretical explanation the paper provides.
The paper formalizes "not breaking clean behavior" as consistency. There are two types:
Definition: The overall accuracy on clean data remains high after backdoor injection.
Global Consistency = |Accuracy_clean - Accuracy_backdoored| is small
Analogy: A student who scores 90% on average before and after learning a bad habit β globally consistent.
Limitation: This is a coarse measure. You could maintain 90% accuracy while completely changing predictions on specific examples.
Definition: For each individual clean input, the model's prediction (and confidence) remains the same after backdoor injection.
For every clean input x:
Output_clean(x) β Output_backdoored(x)
Analogy: The student gets the same questions right AND wrong β not just the same overall score.
Consider this scenario:
Before backdoor injection:
Image A β "Cat" (confidence 99%)
Image B β "Dog" (confidence 95%)
Image C β "Bird" (confidence 88%)
Overall accuracy: 100%
After backdoor injection:
Image A β "Dog" (confidence 60%) β CHANGED!
Image B β "Cat" (confidence 70%) β CHANGED!
Image C β "Bird" (confidence 85%)
Overall accuracy: 33%... wait, or maybe still 100% if labels match
Even if global accuracy stays the same, individual predictions flipping is suspicious and unstable.
Key Point: Instance-wise consistency is strictly stronger than global consistency. If you achieve instance-wise consistency, global consistency follows automatically.
Before a neural network outputs probabilities, it produces raw scores called logits:
Input Image β [Neural Network] β Logits β Softmax β Probabilities
Example logits: [2.1, 0.3, -1.5] (for classes: Cat, Dog, Bird)
After softmax: [0.80, 0.16, 0.04] (probabilities)
Anchor (freeze/pin) the logit outputs on clean data to match the original clean model:
Anchoring Goal:
f_backdoored(x_clean) β f_clean(x_clean) [logit level]
This is stronger than just matching the final class label β it matches the entire output distribution.
The total training loss becomes:
L_total = L_backdoor + Ξ» Β· L_anchor
Where:
| Term | Purpose |
|---|---|
L_backdoor | Makes the model learn the backdoor trigger |
L_anchor | Keeps clean data outputs close to original model |
Ξ» | Balance parameter (how much to weight anchoring) |
The Anchoring Loss specifically:
L_anchor = Ξ£ βf_backdoored(x_clean) - f_clean(x_clean)βΒ²
for all clean samples x_clean
This is essentially Mean Squared Error between logit vectors.
WITHOUT anchoring:
Clean model β Fine-tune for backdoor β Backdoored model
β
Clean behavior drifts β
WITH anchoring:
Clean model β Fine-tune for backdoor β Backdoored model
β β
Logits ββββββ Anchoring Loss βββββββββ Logits
(frozen reference) (must stay close) β
The paper provides a theoretical bound showing:
If the anchoring loss
L_anchoris small (β€ Ξ΅), then the instance-wise consistency is guaranteed to be within a bounded error.
Intuition behind the proof:
βf_backdoored(x) - f_clean(x)β β€ Ξ΅This gives the method a mathematical guarantee β not just empirical hope.
PROBLEM:
Inject backdoor into clean model without detection
CHALLENGE:
Fine-tuning breaks clean data behavior (consistency)
OBSERVATION:
Weight changes during backdoor injection are always AWPs
(small but impactful perturbations)
FORMALIZATION:
- Global Consistency: overall accuracy preserved
- Instance-Wise Consistency: per-sample outputs preserved
SOLUTION:
Logit Anchoring Loss β penalize deviation from clean model's
logit outputs on clean data
RESULT:
Better consistency (especially instance-wise) with theoretical guarantee
| Concept | Definition | Importance |
|---|---|---|
| Backdoor Attack | Hidden malicious behavior triggered by specific input pattern | Core attack mechanism |
| AWP | Small weight changes with large behavioral impact | Explains why backdoor injection is feasible |
| Global Consistency | Overall clean accuracy maintained | Necessary but insufficient |
| Instance-Wise Consistency | Per-sample predictions maintained | Stronger, more stealthy |
| Logit Anchoring | Loss term pinning clean-data logits to original model | The proposed solution |
| Theoretical Guarantee | Bounded consistency error when anchoring loss is small | Formal validation |
Bottom Line: This paper shows that when you fine-tune a clean model to add a backdoor, the weight changes are naturally small (AWPs). To exploit this and maintain stealthy behavior, they propose logit anchoring β a loss that forces the backdoored model to produce identical logit outputs on clean data, with provable consistency guarantees.