Step-by-Step Study Guide

Step 1: Understand What Internal AI Agents Are

What is an Internal Agent?

An internal AI agent is a software bot that:

• Answers questions from employees
• Takes autonomous actions (creates tickets, builds demos)
• Integrates with existing tools (Slack, Salesforce, Linear)

Think of it this way:

Employee asks question in Slack
        ↓
Agent receives request
        ↓
Agent searches knowledge base / database
        ↓
Agent responds OR takes action

---

Step 2: Learn the Three Core Use Cases

Use Case 1: Sales Support

Key insight: Agents don't just answer — they create deliverables (demos) on demand.

---

Use Case 2: Support & Engineering Triage

The triage bot automates ticket creation by following this workflow:

Bug/Feature Request Submitted
        ↓
Search Slack for relevant context
        ↓
Check for duplicate tickets
        ↓
Verify enough detail exists
        ↓
Open Linear ticket with correct priority
        ↓
Route to correct team (via ownership document)
        ↓
Verify ticket fields are correct

Result: 150 tickets created in one month — previously done manually.

---

Use Case 3: Data Analysis

• Product teams ask data questions in domain-specific channels
• Agent returns analysis in the same thread
• Teammates can ask follow-up questions conversationally

Example question:

"How has free-to-paid conversion changed over 6 months in the US? What are the main drivers?"

---

Step 3: Understand Why Reliability Matters First

The Trust Problem

Agent makes early mistakes
        ↓
Team sees errors in shared channels
        ↓
Team loses trust
        ↓
Team stops using the agent
        ↓
Agent fails to deliver value

The Success Flip Side

Successes are equally visible in shared channels — which drives adoption when the agent works well.

Core principle: Get accuracy right before scaling usage.

---

Step 4: Master the Four Best Practices

Best Practice 1: Schema Curation Over Prompt Engineering

Why it works:

• Every extra column = another opportunity for the model to guess wrong
• Fewer fields = fewer errors = higher accuracy

✅ Action: Cut your schema to only the critical fields needed.

---

Best Practice 2: Picklist Typings

The Problem:

• Fields like "deal stage" have fixed valid values
• AI may generate a plausible but non-existent value

The Solution:

Fetch valid picklist values at startup
        ↓
Inject them into the tool's type definitions
        ↓
Model can ONLY select real, valid options

✅ Action: Always constrain fields with fixed values to their actual valid options.

---

Best Practice 3: Prune Procedures Regularly

The Problem:

• As you add procedures over time, they begin to overlap
• Overlapping logic creates conflicting behavior

The Solution:

• Periodically review and restructure procedures
• Ensure each procedure has a clear, distinct trigger
• Eliminate conflicting logic

✅ Action: Audit procedures regularly — remove or merge overlapping ones.

---

Best Practice 4: Task Verification

The Problem:

• Agents occasionally hallucinate field values
• Tickets may route to the wrong team

The Solution:

Agent completes task (e.g., creates ticket)
        ↓
Agent verifies fields are set correctly
        ↓
Agent confirms correct team assignment
        ↓
Flags or corrects errors before finalizing

✅ Action: Build a verification step after every consequential action.

---

Step 5: Connect It All Together

The Big Picture Framework

INTERNAL AI AGENT SUCCESS MODEL

1. IDENTIFY USE CASE
   └── Information retrieval? Action-taking? Data analysis?

2. CURATE DATA CAREFULLY
   └── Limit schema fields → higher accuracy

3. CONSTRAIN VALID INPUTS
   └── Picklist typings → no hallucinated values

4. BUILD CLEAN PROCEDURES
   └── No overlapping logic → predictable behavior

5. VERIFY ACTIONS
   └── Check outputs → catch errors before they spread

6. DEPLOY IN SHARED CHANNELS
   └── Visibility drives trust AND adoption

---

Quick Summary Table

---

Self-Check Questions

1. What is the difference between an agent that retrieves information vs. one that takes action?
2. Why does removing schema fields improve accuracy more than prompt tuning?
3. What problem do picklist typings solve?
4. Why is deploying in a shared channel both a risk and an advantage?
5. At what point in a workflow should task verification occur?