How FairVFL Makes Federated Learning Fair and Private

Peter Bubenik · Sony AI · · Source

Learning Outcomes

By the end of this study, you should be able to:

  1. Define Vertical Federated Learning (VFL) and explain how it differs from standard centralized ML
  2. Explain why fairness is a challenge in VFL settings
  3. Describe the FairVFL framework architecture and its core components
  4. Understand how adversarial learning removes bias from unified representations
  5. Explain how contrastive adversarial learning protects privacy in FairVFL
  6. Evaluate why existing fair ML methods fail in federated scenarios

Step 1: Foundations — What is Federated Learning?

Standard Machine Learning (Centralized)

In traditional ML, all data is collected in one central location:

[User Data] ──→ [Central Server] ──→ [Train Model]

Problem: Sharing raw data raises serious privacy concerns.

Federated Learning (FL)

FL allows model training without sharing raw data:

[Platform A] ──→ [Local Model] ──┐
                                  ├──→ [Server aggregates] ──→ [Global Model]
[Platform B] ──→ [Local Model] ──┘

Horizontal vs. Vertical FL

TypeWhat's distributed?Example
Horizontal FLSame features, different usersTwo hospitals with same patient records
Vertical FL (VFL)Different features, same usersBank has income data; App has behavior data

🔑 Key Insight: In VFL, different platforms hold different feature columns about the same set of users.


Step 2: Understanding the Fairness Problem in VFL

What is Fairness in ML?

A model is unfair if it makes biased predictions based on sensitive attributes like:

  • Gender
  • Race
  • Age

Why Does Bias Occur?

Real-world training data often reflects historical biases:

Training Data (biased) ──→ Model learns bias ──→ Unfair predictions

Example: A loan approval model trained on historically biased data may unfairly deny loans to women.

The VFL Fairness Challenge

Here's the critical problem:

Platform A (Bank):          Platform B (App):
[Income, Credit Score]      [Gender, Age, Behavior]
     ↑                            ↑
Fairness-INSENSITIVE         Fairness-SENSITIVE
features                     features (Gender, Age)

Existing fair ML methods require:

  • Centralized access to sensitive features (gender, race)
  • Direct manipulation of sensitive attributes during training

In VFL, this is impossible because:

  • Sensitive features live on a separate platform
  • Platforms cannot share raw data with each other
  • Privacy regulations prevent direct feature access

🔑 Key Insight: You cannot simply "remove bias" if you cannot directly see or access the sensitive features.


Step 3: The FairVFL Framework — Big Picture

FairVFL solves this with a three-stage pipeline:

┌─────────────────────────────────────────────────────────┐
│                    FairVFL Pipeline                      │
│                                                         │
│  Stage 1: Local Representation Learning                 │
│  Platform A → learns local representation rA            │
│  Platform B → learns local representation rB            │
│                        ↓                               │
│  Stage 2: Unified Representation (Server)               │
│  Server aggregates: r = Aggregate(rA, rB)               │
│                        ↓                               │
│  Stage 3: Fair + Private Unified Representation         │
│  Apply adversarial learning to remove bias              │
│  Apply contrastive learning to protect privacy          │
└─────────────────────────────────────────────────────────┘

Step 4: Stage 1 — Local Representation Learning

Each platform independently learns a local representation from its own features.

Platform A (insensitive features):
[Income=50k, Credit=700] ──→ [Encoder A] ──→ rA (vector)

Platform B (sensitive features):
[Gender=F, Age=28, Clicks=42] ──→ [Encoder B] ──→ rB (vector)

What is a Representation?

A representation (or embedding) is a compressed, learned vector that captures the important information from raw features.

Raw Features → Neural Network Encoder → Dense Vector (representation)
[50000, 700]  →      [Encoder]        →  [0.3, -0.7, 0.1, ...]

Privacy preserved: Only the representations (vectors) are shared, NOT the raw features.


Step 5: Stage 2 — Unified Representation on Server

The server receives local representations and aggregates them:

Server receives:
  rA from Platform A
  rB from Platform B

Aggregation:
  r_unified = Aggregate(rA, rB)
  (e.g., concatenation or weighted sum)

r_unified is used for the target task (e.g., loan prediction)

The Problem at This Stage

Even though raw data isn't shared, the unified representation still carries bias:

rB encodes gender information
     ↓
r_unified inherits gender bias
     ↓
Model predictions become unfair

Step 6: Stage 3a — Adversarial Learning for Fairness

What is Adversarial Learning?

Adversarial learning uses two competing networks:

┌─────────────────────────────────────────┐
│         Adversarial Setup               │
│                                         │
│  [Representation] → [Main Model]        │
│        ↓                                │
│  [Adversary] tries to predict           │
│   sensitive attribute (gender)          │
│                                         │
│  Goal: Make adversary FAIL              │
│  → Representation contains NO           │
│    gender information                   │
└─────────────────────────────────────────┘

How It Works in FairVFL

Step 1: Server sends r_unified to Platform B (which has sensitive features)

Step 2: Platform B runs an adversary:
        Adversary tries to predict gender from r_unified

Step 3: Two competing objectives:
        - Main model: Maximize prediction accuracy (loan approval)
        - Adversary: Maximize ability to predict gender

Step 4: Training signal sent back to server:
        - Penalize r_unified if gender can be predicted from it
        - Force encoder to remove gender information

Result: r_unified becomes FAIR (gender-neutral)

Mathematical Intuition

Loss = Task_Loss - λ × Adversary_Loss

Where:
  Task_Loss    = how well we predict the target (loan approval)
  Adversary_Loss = how well adversary predicts gender
  λ            = fairness-accuracy trade-off weight

Minimizing this loss:
  ✓ Keeps task performance high
  ✓ Makes gender prediction hard (fairness)

Step 7: Stage 3b — Contrastive Adversarial Learning for Privacy

The New Privacy Problem

When the server sends r_unified to Platform B for adversarial training, a new risk emerges:

r_unified contains information from Platform A's data
     ↓
Platform B could reverse-engineer Platform A's private data
     ↓
Privacy violation!

Solution: Contrastive Adversarial Learning

FairVFL adds a privacy protection step before sending r_unified to Platform B.

What is Contrastive Learning?

Contrastive learning teaches a model to:

  • Pull similar samples closer in representation space
  • Push different samples apart
Similar users:    [r1] ←──close──→ [r2]
Different users:  [r1] ←──far───→ [r3]

How Contrastive Adversarial Learning Works

Step 1: Server creates a "privacy-scrubbed" version of r_unified
        called r_private

Step 2: Contrastive objective:
        - r_private should be SIMILAR to r_unified 
          (preserves fairness-relevant info)
        - r_private should be DIFFERENT from raw Platform A features
          (removes private info)

Step 3: An adversary on the server tries to reconstruct 
        Platform A's private data from r_private

Step 4: Training forces r_private to fool this adversary
        → Private information is removed

Step 5: Only r_private (not r_unified) is sent to Platform B

Visual Summary

r_unified (has privacy info)
     ↓
[Contrastive Adversarial Module on Server]
     ↓
r_private (privacy removed, fairness info kept)
     ↓
Sent to Platform B for adversarial fairness training

Step 8: Complete FairVFL Architecture

┌──────────────────────────────────────────────────────────────────┐
│                    COMPLETE FairVFL FLOW                         │
│                                                                  │
│  Platform A              SERVER              Platform B          │
│  (insensitive)                               (sensitive)         │
│                                                                  │
│  [Features A]                                [Features B]        │
│      ↓                                           ↓              │
│  [Encoder A] ──→ rA ──→ [Aggregator] ←── rB ←── [Encoder B]    │
│                              ↓                                   │
│                         r_unified                                │
│                              ↓                                   │
│                    [Contrastive Adversarial]                     │
│                    (removes privacy info)                        │
│                              ↓                                   │
│                         r_private ──────────────→ [Adversary]   │
│                                                   (removes bias) │
│                              ↓                        ↓          │
│                         [Task Model]          Gradient feedback  │
│                              ↓                                   │
│                         Prediction                               │
└──────────────────────────────────────────────────────────────────┘

Step 9: Why This Works — Key Innovations Summary

ChallengeFairVFL Solution
Bias in VFL modelsAdversarial learning on unified representations
Sensitive features on separate platformSend representations (not raw data) to sensitive platform
Privacy risk when sharing representationsContrastive adversarial learning removes private info first
Decentralized featuresLocal encoders + server aggregation

Step 10: Putting It All Together — Conceptual Review

Scenario Walkthrough

Setup: Bank (Platform A) has financial features. Social media app (Platform B) has demographic features including gender. Goal: Predict creditworthiness fairly.

  1. Bank encodes financial data → sends rA to server
  2. App encodes demographic data → sends rB to server
  3. Server aggregates → r_unified
  4. Server applies contrastive adversarial learning → r_private (gender info scrubbed for privacy)
  5. Server sends r_private to App (Platform B)
  6. App runs adversary: tries to predict gender from r_private
  7. Gradient flows back to server: penalize any remaining gender signal
  8. Result: Fair predictions without exposing raw data from either platform

Quick Reference: Key Terms

TermDefinition
VFLFederated learning where different platforms hold different features of same users
Fairness-sensitive featuresAttributes like gender/race that should not influence predictions
Local representationCompressed vector learned from local features by each platform
Unified representationServer-side aggregation of all local representations
Adversarial learningTwo competing networks; one removes unwanted information
Contrastive learningLearning by comparing similar vs. different samples
Contrastive adversarial learningCombination that removes private info while preserving useful structure

Self-Assessment Questions

  1. Why can't existing fair ML methods be directly applied to VFL?
  2. What information is shared between platforms in FairVFL? What is NOT shared?
  3. What two competing objectives exist in adversarial fairness training?
  4. Why is contrastive adversarial learning needed even after adversarial fairness training?
  5. How does FairVFL achieve both fairness and privacy simultaneously?

More to study