After studying this material, you should be able to:

1. Explain what Federated Learning is and why it's vulnerable to attacks
2. Describe how Byzantine defenses work and their common assumptions
3. Define gradient skew and explain why it occurs
4. Analyze how gradient skew creates a vulnerability in Byzantine defenses
5. Understand how the STRIKE attack exploits this vulnerability

---

Step-by-Step Study Material

Step 1: Foundation — What is Federated Learning (FL)?

The Basic Concept

Traditional ML:          Federated Learning:
                         
[All Data] → [Server]    [Client 1 Data] → [Client 1] ──┐
                         [Client 2 Data] → [Client 2] ──┤→ [Server]
                         [Client 3 Data] → [Client 3] ──┘

Federated Learning is a distributed machine learning approach where:

• Multiple clients (e.g., phones, hospitals) train locally on their own data
• Clients send gradients (not raw data) to a central server
• The server aggregates gradients to update a global model
• Raw data never leaves the client → privacy preserved

Why Gradients?

A gradient tells the model: "Here's which direction to adjust your parameters to reduce error."

Each client computes:

gradient_i = ∇Loss(model, local_data_i)

The server aggregates:

global_update = Aggregate(gradient_1, gradient_2, ..., gradient_n)

---

Step 2: The Byzantine Threat

What is a Byzantine Attack?

A Byzantine client is a malicious participant who sends corrupted or fake gradients to sabotage the global model.

Honest clients:    gradient_1 ✓
                   gradient_2 ✓
                   gradient_3 ✓
Byzantine client:  gradient_4 ✗ ← crafted to mislead the server

Goal of attacker: Make the global model perform poorly (e.g., misclassify images, fail predictions)

Why is FL Vulnerable?

The server cannot verify whether a gradient is honest — it just receives numbers.

---

Step 3: Byzantine Defenses — The Common Strategy

The Core Assumption (Inductive Bias)

Most defenses are built on this belief:

🔑 "Gradients that cluster together (densely distributed) are more likely to be honest."

How Defenses Use This

They filter gradients by looking at their distribution in space:

Gradient Space:

    ●●●●●    ← Dense cluster → TRUSTED (assumed honest)
    ●●●●
    
         ×   ← Outlier → REJECTED (assumed Byzantine)

Common Defense Methods

All share the same bias: Dense = Honest, Outlier = Malicious

---

Step 4: The Key Discovery — Gradient Skew

What is Gradient Skew?

This is the central new concept in the paper.

Gradient Skew: When clients have heterogeneous (non-identical) data, a dense cluster of honest gradients can systematically point away from the true optimal direction.

Why Does This Happen?

In real FL, data is non-IID (not identically distributed):

Client 1: Mostly cats 🐱
Client 2: Mostly cats 🐱
Client 3: Mostly cats 🐱
Client 4: Mostly dogs 🐶
Client 5: Mostly dogs 🐶

The optimal gradient = average of ALL honest gradients (what we want):

Optimal = (grad_1 + grad_2 + grad_3 + grad_4 + grad_5) / 5

But the dense cluster (clients 1, 2, 3) points strongly toward "cat features":

Dense cluster direction → biased toward cats 🐱
Optimal direction       → balanced between cats and dogs

Visual Representation

                    ↑ Dense cluster gradients
                   /  (skewed, biased toward majority class)
                  /
                 /___→ Optimal gradient (true average)
                
        The ANGLE between them = GRADIENT SKEW

Key Insight

Dense cluster ≠ Optimal direction

The dense cluster is skewed away from where the model should actually go.

---

Step 5: Why Gradient Skew Breaks Byzantine Defenses

The Vulnerability Created

Because the dense cluster is skewed, there is now space near the dense cluster that is:

• ✅ Close to the dense cluster (looks honest to defenses)
• ❌ Far from the optimal gradient (actually harmful)

Gradient Space:

    ●●●●●  ×  ← Byzantine gradient hides HERE
    ●●●●       (near dense cluster, far from optimal)
    
         ★    ← Optimal gradient (ignored by defense)

The Defense Gets Confused

Defense logic:  "× is near the dense cluster → must be honest ✓"
Reality:        "× is pulling the model away from optimal → harmful ✗"

The Byzantine attacker exploits the skew as a hiding spot.

---

Step 6: The STRIKE Attack

What is STRIKE?

STRIKE = Skew-aware aTtack to cIrcumvent Byzantine defeRses in federated Learning with Exploitation

A two-phase attack that exploits gradient skew:

Phase 1: Find the Skewed Gradients

Step 1: Observe the gradient distribution
Step 2: Identify the dense cluster
Step 3: Measure the skew direction
        (where the cluster points vs. where optimal is)

The attacker identifies the skewed region — the area near the dense cluster but away from optimal.

Phase 2: Construct Byzantine Gradients

Step 1: Craft gradients that:
        ✓ Fall within the dense cluster (pass defense filters)
        ✓ Point away from optimal (cause model degradation)
        
Step 2: Submit these gradients to the server

Attack Flow Diagram

[Attacker observes gradients]
         ↓
[Identifies dense cluster + skew direction]
         ↓
[Crafts Byzantine gradients inside skewed region]
         ↓
[Submits to server]
         ↓
[Defense accepts them as honest]
         ↓
[Global model degrades]

---

Step 7: Experimental Validation

What Was Tested?

The paper validates STRIKE on 3 benchmark datasets (common in FL research, e.g., MNIST, CIFAR-10, Shakespeare)

What Results Show

---

Summary: The Complete Picture

FEDERATED LEARNING ECOSYSTEM:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

1. SETUP:
   Heterogeneous clients → Non-IID data → Gradient Skew

2. DEFENSE ASSUMPTION (flawed):
   Dense cluster = Honest gradients

3. REALITY (discovered in paper):
   Dense cluster ≠ Optimal direction (due to skew)
   
4. ATTACK (STRIKE):
   Hide Byzantine gradients inside skewed dense cluster
   
5. RESULT:
   Defense is fooled → Model is compromised

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---

Key Takeaways

🔑 Core Lesson: Security assumptions that seem reasonable (dense = honest) can be fundamentally broken when the underlying data distribution is heterogeneous — a common real-world condition in Federated Learning.