By the end of this study, you should be able to:
In traditional ML, all data is collected in one central location:
[User Data] ──→ [Central Server] ──→ [Train Model]
Problem: Sharing raw data raises serious privacy concerns.
FL allows model training without sharing raw data:
[Platform A] ──→ [Local Model] ──┐
├──→ [Server aggregates] ──→ [Global Model]
[Platform B] ──→ [Local Model] ──┘
| Type | What's distributed? | Example |
|---|---|---|
| Horizontal FL | Same features, different users | Two hospitals with same patient records |
| Vertical FL (VFL) | Different features, same users | Bank has income data; App has behavior data |
🔑 Key Insight: In VFL, different platforms hold different feature columns about the same set of users.
A model is unfair if it makes biased predictions based on sensitive attributes like:
Real-world training data often reflects historical biases:
Training Data (biased) ──→ Model learns bias ──→ Unfair predictions
Example: A loan approval model trained on historically biased data may unfairly deny loans to women.
Here's the critical problem:
Platform A (Bank): Platform B (App):
[Income, Credit Score] [Gender, Age, Behavior]
↑ ↑
Fairness-INSENSITIVE Fairness-SENSITIVE
features features (Gender, Age)
Existing fair ML methods require:
In VFL, this is impossible because:
🔑 Key Insight: You cannot simply "remove bias" if you cannot directly see or access the sensitive features.
FairVFL solves this with a three-stage pipeline:
┌─────────────────────────────────────────────────────────┐
│ FairVFL Pipeline │
│ │
│ Stage 1: Local Representation Learning │
│ Platform A → learns local representation rA │
│ Platform B → learns local representation rB │
│ ↓ │
│ Stage 2: Unified Representation (Server) │
│ Server aggregates: r = Aggregate(rA, rB) │
│ ↓ │
│ Stage 3: Fair + Private Unified Representation │
│ Apply adversarial learning to remove bias │
│ Apply contrastive learning to protect privacy │
└─────────────────────────────────────────────────────────┘
Each platform independently learns a local representation from its own features.
Platform A (insensitive features):
[Income=50k, Credit=700] ──→ [Encoder A] ──→ rA (vector)
Platform B (sensitive features):
[Gender=F, Age=28, Clicks=42] ──→ [Encoder B] ──→ rB (vector)
A representation (or embedding) is a compressed, learned vector that captures the important information from raw features.
Raw Features → Neural Network Encoder → Dense Vector (representation)
[50000, 700] → [Encoder] → [0.3, -0.7, 0.1, ...]
✅ Privacy preserved: Only the representations (vectors) are shared, NOT the raw features.
The server receives local representations and aggregates them:
Server receives:
rA from Platform A
rB from Platform B
Aggregation:
r_unified = Aggregate(rA, rB)
(e.g., concatenation or weighted sum)
r_unified is used for the target task (e.g., loan prediction)
Even though raw data isn't shared, the unified representation still carries bias:
rB encodes gender information
↓
r_unified inherits gender bias
↓
Model predictions become unfair
Adversarial learning uses two competing networks:
┌─────────────────────────────────────────┐
│ Adversarial Setup │
│ │
│ [Representation] → [Main Model] │
│ ↓ │
│ [Adversary] tries to predict │
│ sensitive attribute (gender) │
│ │
│ Goal: Make adversary FAIL │
│ → Representation contains NO │
│ gender information │
└─────────────────────────────────────────┘
Step 1: Server sends r_unified to Platform B (which has sensitive features)
Step 2: Platform B runs an adversary:
Adversary tries to predict gender from r_unified
Step 3: Two competing objectives:
- Main model: Maximize prediction accuracy (loan approval)
- Adversary: Maximize ability to predict gender
Step 4: Training signal sent back to server:
- Penalize r_unified if gender can be predicted from it
- Force encoder to remove gender information
Result: r_unified becomes FAIR (gender-neutral)
Loss = Task_Loss - λ × Adversary_Loss
Where:
Task_Loss = how well we predict the target (loan approval)
Adversary_Loss = how well adversary predicts gender
λ = fairness-accuracy trade-off weight
Minimizing this loss:
✓ Keeps task performance high
✓ Makes gender prediction hard (fairness)
When the server sends r_unified to Platform B for adversarial training, a new risk emerges:
r_unified contains information from Platform A's data
↓
Platform B could reverse-engineer Platform A's private data
↓
Privacy violation!
FairVFL adds a privacy protection step before sending r_unified to Platform B.
Contrastive learning teaches a model to:
Similar users: [r1] ←──close──→ [r2]
Different users: [r1] ←──far───→ [r3]
Step 1: Server creates a "privacy-scrubbed" version of r_unified
called r_private
Step 2: Contrastive objective:
- r_private should be SIMILAR to r_unified
(preserves fairness-relevant info)
- r_private should be DIFFERENT from raw Platform A features
(removes private info)
Step 3: An adversary on the server tries to reconstruct
Platform A's private data from r_private
Step 4: Training forces r_private to fool this adversary
→ Private information is removed
Step 5: Only r_private (not r_unified) is sent to Platform B
r_unified (has privacy info)
↓
[Contrastive Adversarial Module on Server]
↓
r_private (privacy removed, fairness info kept)
↓
Sent to Platform B for adversarial fairness training
┌──────────────────────────────────────────────────────────────────┐
│ COMPLETE FairVFL FLOW │
│ │
│ Platform A SERVER Platform B │
│ (insensitive) (sensitive) │
│ │
│ [Features A] [Features B] │
│ ↓ ↓ │
│ [Encoder A] ──→ rA ──→ [Aggregator] ←── rB ←── [Encoder B] │
│ ↓ │
│ r_unified │
│ ↓ │
│ [Contrastive Adversarial] │
│ (removes privacy info) │
│ ↓ │
│ r_private ──────────────→ [Adversary] │
│ (removes bias) │
│ ↓ ↓ │
│ [Task Model] Gradient feedback │
│ ↓ │
│ Prediction │
└──────────────────────────────────────────────────────────────────┘
| Challenge | FairVFL Solution |
|---|---|
| Bias in VFL models | Adversarial learning on unified representations |
| Sensitive features on separate platform | Send representations (not raw data) to sensitive platform |
| Privacy risk when sharing representations | Contrastive adversarial learning removes private info first |
| Decentralized features | Local encoders + server aggregation |
Setup: Bank (Platform A) has financial features. Social media app (Platform B) has demographic features including gender. Goal: Predict creditworthiness fairly.
rA to serverrB to serverr_unifiedr_private (gender info scrubbed for privacy)r_private to App (Platform B)r_private| Term | Definition |
|---|---|
| VFL | Federated learning where different platforms hold different features of same users |
| Fairness-sensitive features | Attributes like gender/race that should not influence predictions |
| Local representation | Compressed vector learned from local features by each platform |
| Unified representation | Server-side aggregation of all local representations |
| Adversarial learning | Two competing networks; one removes unwanted information |
| Contrastive learning | Learning by comparing similar vs. different samples |
| Contrastive adversarial learning | Combination that removes private info while preserving useful structure |