Step-by-Step Study Material

---

STEP 1: Understand the Problem Being Solved

Why Did Admiral Need AI Agents?

Before building anything, ask: What was broken?

Key Insight #1

Not every problem needs AI. But when volume, language diversity, and 24/7 demands collide — AI agents become a strategic necessity.

---

STEP 2: Define What "Success" Actually Means

Admiral's North Star Metric

Most companies measure containment — did the AI keep the customer from reaching a human?

Admiral measured something harder and more meaningful:

PRIMARY METRIC → First Contact Resolution (FCR)
Goal: 90% of customers get their problem solved on first contact

SECONDARY METRIC → Containment
(Only tracked after FCR was established)

Why This Distinction Matters

Key Insight #2

A deliberate handoff to a human is NEVER counted as a failure. Escalation ≠ failure. Unresolved customer need = failure.

Practical Example:

• Customer calls about a vulnerable financial situation → AI transfers to human specialist → ✅ SUCCESS
• Customer calls about loan settlement → AI gives wrong answer → ❌ FAILURE

---

STEP 3: Learn the Technical Architecture

The Agent Structure (Settlement Quote Demo)

Admiral didn't build one giant AI. They built specialized sub-agents working together:

INCOMING CALL
     ↓
[SENTRY SUB-AGENT]
Checks: Is this customer eligible for AI handling?
  → Vulnerable customer? → Human priority queue
  → Customer in arrears? → Human priority queue
  → Standard customer? → Continue
     ↓
[AUTHENTICATION SUB-AGENT]
Verifies customer identity before anything else executes
     ↓
[MAIN AGENT]
Handles the actual request (e.g., settlement quote)
API-driven → pulls data directly, no screen waiting
     ↓
[FEEDBACK AGENT]
Asks customer to rate experience 1-5
     ↓
CALL ENDS

Why API-Driven Matters

Key Insight #3

The time savings came from workflow redesign, not just from replacing humans with AI. The architecture change was as important as the AI itself.

---

STEP 4: Understand Compliance in Regulated Industries

The Core Tension

INNOVATION PRESSURE          COMPLIANCE PRESSURE
"Deploy fast"          vs.   "Never harm customers"
"Iterate quickly"      vs.   "Meet regulatory standards"

Admiral's Solution: Raise Validation, Not Lower Compliance

Admiral's standard before going live:

"The agentic solution must be as good as our best human agents, or better — before it goes live."

How They Handle Edge Cases

Old world: Edge case knowledge lived in the instincts of experienced agents (tacit knowledge)

New world: Every edge case must be:

1. Identified explicitly
2. Tested systematically
3. Documented for every use case

Vulnerability Detection — A Non-Negotiable

In insurance, identifying vulnerable customers is a legal and ethical baseline.

Admiral's approach uses redundancy by design:

VULNERABILITY DETECTION SYSTEM

Traditional ML Models  +  Generative AI Models
        ↓                        ↓
        └──────────┬─────────────┘
                   ↓
         If EITHER signal fires
                   ↓
         Route to human specialist

Why both? Neither model alone is trusted enough. Running both means vulnerable customers are caught regardless of which system detects the signal first.

Key Insight #4

Outcome-based regulation is actually flexible. Regulators care about what happens to the customer, not exactly how the technology works. This gives teams room to innovate on the "how" while holding firm on the "what."

---

STEP 5: Master the Deployment and Testing Model

The Olivia (France) Deployment Pipeline

Admiral doesn't push changes directly to live. They use gradual traffic splitting:

NEW CHANGE BUILT ON BRANCH
         ↓
    Simulation Tests
         ↓
    Manual Preview
         ↓
    1% live traffic
         ↓
    2% → 5% → 10% → 25% → 50% → 100%
         ↓
  Data supports change? → Continue
  Data doesn't support? → Instantly back to 0%
                          (Nothing to roll back)

Full cycle time: A few hours to one day Old way: Two-week sprints + two-week deployments = one month per change

The Language Insight

Admiral built Olivia's prompts directly in French, not translated from English.

Why it matters:

English prompt → translated to French = Model reasons in English, outputs French
French prompt written natively = Model reasons in French from the start

Result: Significantly better performance when prompting in the customer's native language.

Key Insight #5

Language is not just a localization detail — it may fundamentally change how the model reasons. This has implications for any global AI deployment.

---

STEP 6: Understand Organizational Change Management

The Three Principles Admiral Used

Admiral's Group Head of GenAI came from McKinsey's change management practice. She applied three principles directly:

---

Principle 1: People Support What They Help Build

❌ Wrong approach: Announce AI is coming, mandate adoption ✅ Right approach: Put the tool in front of skeptics

Admiral ran demos across the organization including:

• Top 50 leaders
• Group CEO
• All built agents from scratch in a training day

Result: Leaders who were skeptical became advocates because they experienced it themselves.

---

Principle 2: It's Never Just About the Technology

The conversation should be about:

P&L Impact → "How does this affect our bottom line?"
Customer Experience → "How does this improve customer outcomes?"
Employee Experience → "How does this change what our people do?"

Not: "Here's a cool AI model we want to deploy."

"Once people align on the outcome, the technology options get figured out."

---

Principle 3: Build Governance Early

---

The Rollout Model

Admiral's approach to scaling across markets:

STEP 1: Choose ONE part of the group
         ↓
STEP 2: Experiment deeply on ONE topic
         ↓
STEP 3: Extract learnings
         ↓
STEP 4: Fast follow in other markets
         ↓
STEP 5: Full rollout after validation

Anti-pattern they avoided: One use case in one domain, repeated from scratch in every market = reinventing the wheel ten times.

---

STEP 7: Learn the Organizational Structure

Hub and Spoke Model

                    [HUB - Group Level]
                   /                   \
        Value Capture Side         Technical Side
        - Strategy                 - Engineers
        - Roadmaps                 - Architects
        - Product Design           - Delivery
        - A/B Testing              - Standards & Integration
              |
    __________|__________
    |         |         |
[SPOKE]   [SPOKE]   [SPOKE]
 UK        France    Spain/Italy
 Local     Local     Local
 experts   experts   experts

The Pod Structure for Building

Each project uses small cross-functional pods:

Process Expert + Business Owner + Product + Tech
                      ↓
              Build together
                      ↓
              Test together
                      ↓
         Give to real customers to break

Key Insight #6

The AI engineer of the future writes less code and does more of: understanding how GenAI works, structuring ideas properly, and leveraging agentic coding tools to deliver.

---

STEP 8: Understand the Global Deployment Challenge

What Changes Market to Market

The Deployment Unit That Works

Engineer (knows prompting + architecture)
              +
Local Business Owner (knows market context)
              =
Effective Deployment Unit

Customer Trust Is the Real Bottleneck

"If your customers don't trust it, if your customers don't understand it, they will immediately — in second number two of the conversation — say 'please transfer me to a human.'"

Admiral expects to spend more time on customer education than on the model itself:

• Welcome messages
• IVR positioning
• Onboarding language
• Setting expectations upfront

---

STEP 9: Understand the Future Vision

Admiral's Roadmap Goal

TODAY:                          FUTURE:
Customer repeats themselves     Customer NEVER repeats themselves
across every channel            
                                Intent carried across channels
IVR → Voice Agent → Human       Sentiment carried across channels
= Start over each time          Vulnerability signals carried across
                                every handoff — not just transcript

The Cost Reality of AI Deployment

Admiral's McKinsey-derived rule of thumb:

£1 spent on technology
£3 spent on process redesign
£5 spent on change management

Implication: If your AI budget is 90% technology and 10% everything else, you are likely to fail.

---

Summary: The Complete Framework

┌─────────────────────────────────────────────────────┐
│           ADMIRAL'S AI AGENT FRAMEWORK              │
├─────────────────────────────────────────────────────┤
│ 1. DEFINE SUCCESS CORRECTLY                         │
│    → FCR first, containment second                  │
├─────────────────────────────────────────────────────┤
│ 2. ARCHITECT FOR SAFETY                             │
│    → Sub-agents, sentry layers, redundant detection │
├─────────────────────────────────────────────────────┤
│ 3. HOLD THE COMPLIANCE BAR                          │
│    → Raise validation, never lower compliance       │
├─────────────────────────────────────────────────────┤
│ 4. DEPLOY GRADUALLY                                 │
│    → Traffic splits, branch testing, fast rollback  │
├─────────────────────────────────────────────────────┤
│ 5. MANAGE CHANGE DELIBERATELY                       │
│    → Co-build, outcome focus, early governance      │
├─────────────────────────────────────────────────────┤
│ 6. SCALE THROUGH HUB AND SPOKE                      │
│    → Central standards + local market expertise     │
├─────────────────────────────────────────────────────┤
│ 7. INVEST IN CUSTOMER TRUST                         │
│    → Education > model sophistication               │
└─────────────────────────────────────────────────────┘

---

Self-Check Questions

Test your understanding:

1. Why did Admiral choose First Contact Resolution as their primary metric instead of containment?
2. What does the sentry sub-agent do, and why does it exist?
3. Why is prompting in the customer's native language better than translating from English?
4. What is the "hub and spoke" model and what problem does it solve?
5. Why does Admiral run both traditional ML and generative AI for vulnerability detection simultaneously?
6. According to the McKinsey rule of thumb, if you spend £1 on technology, how much should you spend on change management?
7. What makes a deliberate human handoff different from an AI failure?