Step-by-Step Teaching

Step 1: Understanding the Core Problem

What Makes AI Agents Different from Regular Pipeline Steps?

Traditional pipeline steps execute quickly. AI agents think before they answer — they reason through prompts, call tools, and generate responses. This takes time, often many seconds.

Traditional step:    |--work--|  (fast, predictable)
AI agent step:       |--------thinking--------|  (slow, variable)

Why Does This Create a Cost Problem?

When you call an agent from a Lambda function the naive way:

Lambda starts → calls agent → WAITS → agent responds → Lambda finishes
     ↑                           ↑
  Billing starts            Billing continues

Key insight: Lambda bills you for every second it runs — including seconds spent doing absolutely nothing while waiting.

The Two-Sided Billing Reality

The waste is NOT on the agent side. It's the caller sitting idle on an open connection.

---

Step 2: The Blocking Anti-Pattern (What NOT to Do)

Lambda Function
┌─────────────────────────────────────────┐
│ 1. Receive request                      │
│ 2. Call agent ──────────────────────┐  │
│ 3. WAIT... (billing continues)      │  │
│ 4. WAIT... (billing continues)      │  │
│ 5. WAIT... (billing continues)      │  │
│ 6. Receive response ◄───────────────┘  │
│ 7. Return result                        │
└─────────────────────────────────────────┘
Billed duration ≈ entire agent processing time

Why it's common: Simple to implement, easy to understand Why it's a problem: Billed duration tracks the agent's runtime directly

---

Step 3: The Fix — Release the Caller During the Wait

The solution in all three patterns follows the same principle:

INSTEAD OF:
Caller → [waits with meter running] → gets result

DO THIS:
Caller → starts agent → returns immediately (meter stops)
         [agent works independently]
         agent → signals completion → pipeline resumes

Now let's look at each pattern.

---

Step 4: Pattern 1 — Task-Token Callback

How It Works

Step Functions
     │
     ▼
Lambda (dispatcher)
  ├── Receives task token from Step Functions
  ├── Starts agent (passes token along)
  └── Returns in ~seconds ← BILLING STOPS HERE
  
[Step Functions execution PAUSES — no compute billed]

Agent works...
Agent finishes...
Agent calls SendTaskSuccess(token)
     │
     ▼
Step Functions RESUMES execution

The Task Token Is the Key

Think of the task token as a claim ticket:

• Step Functions hands it to Lambda
• Lambda passes it to the agent
• The agent uses it to "wake up" the paused execution when done

Real Numbers from Testing

State active:          19.6 seconds
Lambda billed:          4.8 seconds
                       ──────────
Idle time (no billing): ~14.8 seconds  ← money saved

When to Use Pattern 1

✅ You need custom code before or after the agent call ✅ You want Lambda in the path for pre/post-processing logic ✅ You need flexibility around the agent invocation

Trade-offs

---

Step 5: Pattern 2 — Direct Service Integration

How It Works

Step Functions
     │
     └──► Calls AgentCore directly (no Lambda at all)
          [Step Functions pauses — bills per state transition only]
          
Agent works...
Agent finishes...
Response flows directly into next state

Why This Is the Simplest Pattern

There is no Lambda function in the path at all. Step Functions uses its built-in AWS SDK integration to call AgentCore directly. The Validate branch becomes a single Task state.

Before (with Lambda):
Step Functions → Lambda → AgentCore

After (direct integration):
Step Functions ──────────► AgentCore

When to Use Pattern 2

✅ Pure orchestration — no custom code needed around the agent call ✅ You want the absolute lowest complexity ✅ State transformations in Amazon States Language (ASL) are sufficient

Trade-offs

---

Step 6: Pattern 3 — Durable Function

How It Works

Single Lambda Function (durable)
┌────────────────────────────────────────┐
│ context.step("organize", ...)          │ ← runs, suspends
│ context.parallel([...])               │ ← runs, suspends
│ context.waitForCallback(agentCall)    │ ← suspends here
│   [SUSPENDED — not billed]            │
│   [agent works independently]         │
│   [agent calls SendDurableCallback]   │
│ context.step("notify", ...)           │ ← resumes, runs
└────────────────────────────────────────┘

The Key Concept: Checkpoint and Replay

The durable function suspends itself at the waitForCallback point. When the agent signals completion, the function replays from a checkpoint — picking up exactly where it left off.

You write the entire pipeline as sequential code in one function, but it doesn't bill for compute during suspensions.

When to Use Pattern 3

✅ You prefer orchestration as code rather than a state machine definition ✅ Complex async workflows that are easier to express sequentially ✅ You want all pipeline logic in one place

Trade-offs

---

Step 7: Side-by-Side Comparison

BLOCKING (anti-pattern)
├── Lambda alive entire time
├── Billed: full agent processing time
└── Use for: prototypes only

PATTERN 1: Task Token
├── Lambda returns in seconds
├── Billed: dispatch only (~seconds)
├── Custom code: yes (before + after)
└── Use for: custom pre/post-processing

PATTERN 2: Direct Integration  
├── No Lambda at all
├── Billed: zero Lambda cost
├── Custom code: limited (ASL only)
└── Use for: pure orchestration

PATTERN 3: Durable Function
├── Lambda suspends during wait
├── Billed: short execution bursts only
├── Custom code: yes (sequential code)
└── Use for: complex async workflows as code

---

Step 8: Production Best Practices

1. Guard Against Agents That Never Answer

Set TimeoutSeconds on every waitForTaskToken state
→ Execution fails cleanly instead of hanging forever

Set HeartbeatSeconds if agent sends heartbeats
→ Faster detection of a dead agent

2. Use Stable Session IDs Across Retries

sessionId = derived from execution name
→ Retries resume the same agent session
→ Prevents starting fresh on every retry

3. Enable X-Ray Tracing

Tracing: Active on Step Functions + Lambda
→ See exactly: agent thinking time vs. caller waiting time
→ Confirms your pattern actually released compute

4. Right-Size the Dispatcher Function

The dispatcher only: serializes request + invokes endpoint
256 MB memory + 30-second timeout = usually sufficient
Heavy work happens on the agent side — not here

---

Step 9: Decision Framework — Which Pattern Should You Choose?

Do you need custom code around the agent call?
│
├── NO → Pattern 2 (Direct Integration)
│         Simplest, zero Lambda cost
│
└── YES → Do you prefer orchestration as code?
          │
          ├── YES → Pattern 3 (Durable Function)
          │          One function, sequential style
          │
          └── NO → Pattern 1 (Task Token)
                    Step Functions state machine
                    with Lambda dispatcher

---

Summary