How to Build Realistic AI Twins of Mobile Users

Peter Bubenik ยท Amazon Research Publication ยท ยท Source
Image for AgenTwin: An end-to-end framework for building agentic twins of mobile users

๐ŸŽฏ LEARNING OUTCOMES

After studying this material, you should be able to:

  1. Define what an agentic twin is and why it matters in mobile user simulation
  2. Identify the four key limitations in existing user simulation approaches
  3. Explain the four core components of the AgenTwin framework
  4. Understand how joint decision-making differs from isolated decision modeling
  5. Evaluate how AgenTwin is validated and what metrics demonstrate its effectiveness

๐Ÿ“š STEP-BY-STEP TEACHING


STEP 1: Foundation โ€” What Is an Agentic Twin?

Start with the "Why"

Imagine a telecom company serving millions of users. They want to:

  • Predict what a user will click next
  • Understand emotional responses to app changes
  • Test new features without real users

The Problem: You cannot interview millions of people. You need a digital replica that thinks, decides, and behaves like a real user.

Definition

๐Ÿค– Agentic Twin = A software agent that autonomously replicates a specific real user's behavior, decisions, preferences, and emotional patterns โ€” not just statically, but dynamically over time.

Key Distinction

ConceptDescription
Digital TwinA static data mirror of a user
Agentic TwinAn active agent that makes decisions, learns, and evolves like the real user

STEP 2: The Problem Space โ€” 4 Key Limitations in Existing Approaches

Before understanding AgenTwin's solution, you must understand what was broken.


Limitation 1: ๐ŸŽญ Subjective Decision-Making Diversity

What it means: Real users are diverse. Two people shown the same app screen make different choices based on personality, mood, culture, and history.

What existing systems did wrong: They modeled decisions using generic or averaged behavior โ€” missing individual uniqueness.

Analogy:

Imagine training a model on "average human food preference" and then predicting whether you specifically prefer sushi or pizza. It fails because it ignores your subjective profile.


Limitation 2: ๐Ÿ“ฆ Scalable Experience Retention

What it means: Real users remember past experiences. If you had a bad checkout experience last week, it affects your behavior today.

What existing systems did wrong: They couldn't store and retrieve growing user histories efficiently at scale (millions of users ร— thousands of interactions).

Analogy:

A friend who forgets every conversation you've had is not a good replica of you.


Limitation 3: ๐Ÿ”€ Simultaneous Multi-Option Processing

What it means: On a mobile screen, users see multiple options at once (e.g., 5 products in a list) and evaluate them together, not one by one.

What existing systems did wrong: They evaluated options in isolation โ€” scoring option A, then B, then C โ€” missing the comparative nature of real human choice.

Analogy:

Choosing a restaurant from a menu is different from being asked "Do you want pasta? Yes/No" five separate times.


Limitation 4: ๐Ÿ“ฑ Granular Mobile Interaction Fidelity

What it means: Mobile interaction is specific โ€” taps, swipes, scroll depth, back-button usage, screen transitions. These micro-behaviors carry meaning.

What existing systems did wrong: They modeled high-level decisions (e.g., "user bought item") but missed the how โ€” the sequence of micro-interactions leading there.

Analogy:

Knowing someone "arrived at work" tells you less than knowing they took the bus, stopped for coffee, and arrived 10 minutes late.


STEP 3: The AgenTwin Framework โ€” 4 Core Components

Each component directly addresses one of the four limitations above.


Component 1: ๐Ÿง  Adaptive Persona Generation

(Solves: Subjective Decision-Making Diversity)

What it does: Creates a unique, rich psychological and behavioral profile for each simulated user.

How to think about it:

Real User Data โ†’ Persona Profile
                    โ”œโ”€โ”€ Preferences (price sensitivity, brand loyalty)
                    โ”œโ”€โ”€ Personality traits (risk-averse, impulsive)
                    โ”œโ”€โ”€ Demographics
                    โ””โ”€โ”€ Behavioral tendencies

Key word: "Adaptive" The persona is not fixed. It updates as new data about the user becomes available.

โœ… Result: Each twin behaves differently because each has a different persona โ€” just like real users.


Component 2: ๐Ÿ—ƒ๏ธ Self-Evolving Contextual Memory

(Solves: Scalable Experience Retention)

What it does: Gives each twin a memory system that stores past interactions and retrieves relevant ones when making new decisions.

How to think about it:

Past Interaction โ†’ Stored in Memory
                        โ†“
New Situation โ†’ Memory Retrieval โ†’ Informed Decision

Key word: "Self-Evolving" The memory updates automatically โ€” it grows with the user's experience without manual intervention.

Why "contextual"? Not all memories are equally relevant. If you're shopping for shoes, your memory of past shoe purchases matters more than your food delivery history.

โœ… Result: Twins make decisions informed by relevant history, just like real users do.


Component 3: โš–๏ธ Multi-Faceted Behavioral Simulation with Joint Decision-Making

(Solves: Simultaneous Multi-Option Processing)

What it does: Evaluates multiple options simultaneously rather than scoring them one at a time.

Isolated vs. Joint Decision-Making:

ApproachMethodProblem
IsolatedScore A โ†’ Score B โ†’ Score C โ†’ Pick highestIgnores relative comparison
JointScore A, B, C together โ†’ Pick based on comparisonMirrors real human choice

Why does this matter on mobile? Mobile screens show limited content. Users scan and compare what's visible simultaneously.

"Multi-Faceted" means: The simulation captures multiple dimensions of behavior:

  • Rational (price, features)
  • Emotional (brand feeling, aesthetics)
  • Social (reviews, popularity)

โœ… Result: Decision predictions are more accurate because they reflect how humans actually choose.


Component 4: ๐Ÿ“ฒ High-Fidelity Mobile Device Control

(Solves: Granular Mobile Interaction Fidelity)

What it does: Simulates actual mobile interactions at the gesture/action level โ€” not just outcomes.

Examples of what it captures:

User Journey Example:
Open App โ†’ Scroll 3 items โ†’ Tap Item 2 โ†’ Read description 
โ†’ Swipe back โ†’ Tap Item 4 โ†’ Add to cart โ†’ Abandon checkout

Why this matters:

  • Scroll depth tells you what caught attention
  • Back-button usage signals hesitation
  • Time-on-screen signals interest level

โœ… Result: The twin doesn't just predict what a user decides โ€” it replicates how they navigate to that decision.


STEP 4: How the Components Work Together

Here is the end-to-end flow of AgenTwin:

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                   REAL USER DATA                    โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
                       โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚         ADAPTIVE PERSONA GENERATION                 โ”‚
โ”‚   โ†’ Who is this user? What drives their choices?   โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
                       โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚       SELF-EVOLVING CONTEXTUAL MEMORY               โ”‚
โ”‚   โ†’ What has this user experienced before?         โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
                       โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚    MULTI-FACETED BEHAVIORAL SIMULATION              โ”‚
โ”‚   โ†’ How does this user evaluate options jointly?   โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
                       โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚       HIGH-FIDELITY MOBILE DEVICE CONTROL           โ”‚
โ”‚   โ†’ What exact actions does this user take?        โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
                       โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚              AGENTIC TWIN OUTPUT                    โ”‚
โ”‚   Predicted decisions + interaction sequences      โ”‚
โ”‚   + emotional responses + reasoning patterns       โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

STEP 5: Validation โ€” How Do We Know It Works?

AgenTwin was tested in three ways:


Validation 1: ๐Ÿ“Š Quantitative Benchmarks

Tested on public benchmarks and proprietary real-world mobile datasets.

Metrics improved:

MetricWhat It Measures
User Decision Prediction AccuracyDoes the twin predict the right choice?
Robustness to Positional BiasDoes it avoid favoring options just because they appear first/top?
Computational EfficiencyCan it scale to millions of users without excessive cost?

๐Ÿ’ก Positional Bias is important: Many AI systems favor the first option in a list. Real users don't always do this. AgenTwin is more robust against this flaw.


Validation 2: ๐Ÿงฉ Reasoning Analysis

What it checks: Does the twin reason about decisions the way a real human would?

Not just "did it pick the right answer?" but "did it think about it correctly?"


Validation 3: ๐Ÿ’ฌ Emotion Perception Experiments

What it checks: Does the twin reflect appropriate emotional responses to situations?

Example: Does the twin show frustration signals when an app crashes? Does it show enthusiasm for a personalized recommendation?

โœ… This makes AgenTwin a holistic simulacrum โ€” not just behaviorally accurate, but emotionally aligned too.


STEP 6: Big Picture โ€” Why Does This Matter?

Real-World Applications

Use CaseHow AgenTwin Helps
Product TestingTest app changes on millions of twins before real rollout
PersonalizationPredict individual user needs at scale
Customer ExperienceIdentify friction points in user journeys
MarketingSimulate responses to campaigns without A/B testing costs

The Scalability Advantage

Traditional Approach:          AgenTwin Approach:
Survey 1,000 users      โ†’      Simulate 1,000,000 twins
Weeks of data collection โ†’     Instant simulation
Limited behavioral depth โ†’     Full interaction fidelity

๐Ÿ”‘ SUMMARY: Key Concepts at a Glance

ConceptOne-Line Definition
Agentic TwinAn autonomous agent that replicates a real user's behavior dynamically
Adaptive PersonaA unique, evolving psychological profile per user
Contextual MemoryA scalable system that stores and retrieves relevant past experiences
Joint Decision-MakingEvaluating multiple options simultaneously, not in isolation
Mobile FidelitySimulating actual tap/swipe/scroll interactions, not just outcomes
Positional BiasThe tendency to favor options based on screen position, not merit
Holistic ValidationTesting accuracy, reasoning quality, AND emotional alignment

โœ… SELF-CHECK QUESTIONS

  1. What is the difference between a digital twin and an agentic twin?
  2. Why is joint decision-making more realistic than isolated option scoring?
  3. How does self-evolving memory improve simulation quality over time?
  4. Why is positional bias robustness an important metric for mobile simulation?
  5. What does emotion perception validation add that accuracy metrics alone cannot?

๐Ÿ You are now equipped to understand AgenTwin's purpose, architecture, and significance in the field of mobile user simulation.

More to study