Building Production-Ready GenAI for People Analytics

Concept 1: The Exploration-Exploitation Tradeoff (The Core "Bandit" Problem)

What is it?

The multi-armed bandit problem is a classic decision-making framework where you must choose between:

Real-World Analogy

Imagine a doctor choosing between a known treatment (reliable but maybe not optimal) vs. a new experimental drug (unknown outcome, but potentially better).

How it appears in the article

The PXT team faces this constantly:

• Should they exploit existing ML models that work reasonably well?
• Or explore new GenAI/LLM approaches that might work better but require investment?

Known Model (Exploit) ←——————→ New LLM Approach (Explore)
     Safe, predictable              Costly, uncertain, potentially superior

---

Concept 2: The "Costly" Part — Why Knowledge Has a Price

What makes bandits "costly"?

In standard bandit problems, exploration is free. In costly bandits, every observation or experiment has a real cost:

Cost of Knowledge = Time + Compute + Risk + Human Capital

Three Types of Costs in This Context

💰 Computational Cost

Foundation Models (Claude, GPT) → API calls cost money per token
Fine-tuning → GPU hours on SageMaker
Deployment → Latency + infrastructure costs

⏱️ Time Cost

• Prompt engineering iterations
• Model evaluation cycles
• A/B testing with real employees

🎯 Opportunity Cost

• Junior scientists mentored = senior scientist time spent
• Every experiment NOT run = potential insight lost

The Key Tension

You cannot learn everything. Every piece of knowledge has a price. The optimal algorithm decides WHEN the price is worth paying.

---

Concept 3: What Makes an Algorithm "Optimal"?

The Regret Framework

An algorithm is measured by regret — the difference between:

• What you actually achieved
• What you could have achieved with perfect knowledge

Regret = Σ (Best Possible Reward - Actual Reward)
              over all decisions made

Minimizing Regret in Practice

The article describes this through their evaluation frameworks:

Rigorous Evaluation = Measuring:
├── Model Performance (are predictions accurate?)
├── Bias Detection (are we fair across groups?)
├── Safety Metrics (are outputs harmful?)
└── Business Impact (does it actually help?)

Each metric answers: "Are we paying the right price for the right knowledge?"

---

Concept 4: The Three Optimal Strategies

Strategy 1: Upper Confidence Bound (UCB) — "Optimism Under Uncertainty"

Principle: When uncertain, assume the best possible outcome and try it.

In the article's context:

New LLM Approach:
├── Estimated Performance: 75%
├── Uncertainty Bonus: +15% (because we haven't tried it much)
└── UCB Score: 90% → TRY IT

How PXT applies this:

• Few-shot learning = try new approaches with minimal examples first
• Rapid prototyping = quickly test before full commitment
• Innovation mandate = structurally rewarded for trying new things

---

Strategy 2: Thompson Sampling — "Probabilistic Exploration"

Principle: Maintain a probability distribution over what's best, sample from it, act on the sample.

Belief about Model A: [====|====] → 60% chance it's best
Belief about Model B: [===|=====] → 40% chance it's best

→ Randomly pick proportional to belief
→ Update belief after seeing result
→ Repeat

In the article's context: This maps to their Bayesian experimental design:

• They don't know which modeling approach (linear, tree-based, DNN, LLM) is best for a given problem
• They maintain beliefs about each
• They allocate scientist time proportionally to those beliefs

---

Strategy 3: Information-Directed Sampling — "The True Optimal for Costly Bandits"

Principle: Don't just maximize reward. Maximize reward per unit of information cost.

                Expected Reward Gain
IDS Score =  ─────────────────────────
              Information Cost of Action

This is the most sophisticated strategy and directly maps to the article's core mission:

PXT Decision Framework:
├── Option A: Fine-tune existing model
│   ├── Expected Gain: +5% accuracy
│   ├── Cost: 2 weeks + $50K compute
│   └── IDS Score: LOW
│
└── Option B: Prompt engineering on Claude
    ├── Expected Gain: +4% accuracy
    ├── Cost: 3 days + $500
    └── IDS Score: HIGH → DO THIS FIRST

---

Concept 5: The Talent Intelligence Layer as a Bandit System

Putting it all together

The article describes building a "talent intelligence layer" — this IS a costly bandit system at scale:

Talent Intelligence Layer
         │
         ├── Natural Language Understanding
         │   └── Which NLP model best understands job descriptions?
         │       (Bandit: explore different models, exploit best one)
         │
         ├── Network Science
         │   └── Which collaboration signals predict performance?
         │       (Bandit: costly to measure, must choose wisely)
         │
         └── Predictive Modeling
             └── Which features matter for employee wellbeing?
                 (Bandit: each feature costs data collection effort)

The System Learns Continuously

Observe Employee Data
        ↓
Update Beliefs About What Works
        ↓
Choose Next Best Action (Optimal Algorithm)
        ↓
Pay the Cost of Learning
        ↓
Receive Signal (Did it help?)
        ↓
[Repeat — 1 million+ employees]

---

Concept 6: Scaling Optimal Algorithms — The "Million People" Problem

Why scale changes everything

Contextual Bandits (The Upgrade)

Standard bandits ask: "What's the best action overall?"

Contextual bandits ask: "What's the best action given this specific context?"

Context = {
    employee_level: "L5",
    domain: "operations",
    tenure: "2 years",
    location: "fulfillment_center"
}

→ Optimal action for THIS person may differ from global optimum

This is why the article emphasizes:

• Multiple business domains (corporate, operations, safety)
• Multiple analysis levels (individual, group, organizational)
• Multiple model types (linear → LLM)

---

Summary: The Complete Mental Model

THE COSTLY BANDIT FRAMEWORK
═══════════════════════════

PROBLEM:  Make decisions under uncertainty when learning is expensive

MEASURE:  Regret = Gap between actual and optimal outcomes

STRATEGIES:
  UCB              → Be optimistic, explore uncertain options
  Thompson         → Sample proportional to belief
  IDS              → Maximize reward per unit of information cost

SCALE UP:
  Contextual       → Personalize decisions to context
  Hierarchical     → Handle multiple levels simultaneously

APPLIED TO PXT:
  ┌─────────────────────────────────────────────┐
  │  Which AI model helps which employee        │
  │  in which context at what cost              │
  │  to maximize wellbeing and productivity     │
  │  across 1M+ people continuously?            │
  └─────────────────────────────────────────────┘

---

Key Takeaway

The "price of knowledge" is not a problem to eliminate — it's a constraint to optimize around. The best algorithms don't try to learn everything; they learn exactly what's needed, when it's needed, at the lowest possible cost, while still achieving near-optimal outcomes.

This is the scientific foundation underlying everything described in the article — from prompt engineering experiments to mentoring junior scientists to building production ML systems.