How to Secure AI Agents at Every Layer

Learning Outcomes

After studying this material, you should be able to:

1. Explain why AI security is an engineering problem requiring defined controls and accountability
2. Identify the layers of an AI agent stack and their security responsibilities
3. Apply established security principles to AI agent deployments
4. Describe the tools and practices used to test, enforce and verify AI security controls
5. Evaluate how open collaboration strengthens defensive security posture

---

Step 1: Foundation — Why AI Security Is an Engineering Problem

The Core Argument

Security is not a feature you add at the end. It is a structured engineering discipline with:

The Historical Pattern

Internet arrived → New operating conditions
                 → Same core responsibilities endured

Cloud arrived   → New operating conditions
                 → Same core responsibilities endured

AI agents arrive → New operating conditions
                 → Same core responsibilities must endure

The Four Enduring Security Responsibilities

1. Establish identity — Who or what is acting?
2. Control access — What are they allowed to do?
3. Limit exposure — How much damage can occur if something goes wrong?
4. Verify protections — Do the controls actually work?

Key Insight: AI agents do not replace these responsibilities. They create new conditions under which the same responsibilities must be applied.

---

Step 2: Understanding the AI Agent Stack

What Makes AI Agents Different

AI agents can:

• Reason through problems
• Use tools to take real-world actions
• Adapt behavior based on data they encounter

This adaptability is powerful — and dangerous. An agent might encounter malicious instructions in a document and act on them.

The Three Layers of the Agent Stack

┌─────────────────────────────────────┐
│           RUNTIME ENVIRONMENT        │  ← Infrastructure where actions execute
├─────────────────────────────────────┤
│              HARNESS                 │  ← Organizes context, tools, workflows
├─────────────────────────────────────┤
│               MODEL                  │  ← Provides reasoning capabilities
└─────────────────────────────────────┘

Each layer carries its own security responsibilities.

A Concrete Example: The Malicious Document Attack

Follow this scenario step by step:

Step 1: Agent is given permission to UPDATE a customer record
        ↓
Step 2: Agent opens an attached document
        ↓
Step 3: Document contains malicious instructions
        ↓
Step 4: Agent attempts to EXPORT customer data to unauthorized destination
        ↓
Step 5: What should happen?

Correct Security Response:

Critical Rule: Permission to perform Action A does not automatically grant permission to perform Action B, even if they seem related.

---

Step 3: Building Security Into Agent Operations

The Boundary Principle

"A security boundary has to hold even when an agent makes the wrong decision."

This is fundamental. You cannot rely on the agent always reasoning correctly. The environment must enforce limits independently.

Agent Reasoning Layer:    "Should I do this?" (can be manipulated)
                                    ↕
Environment Control Layer: "Are you allowed to do this?" (must be enforced externally)

Five Operational Security Requirements

1. Traceable Identity

• Every agent needs its own identity
• Credentials must be limited to its assigned task
• No shared credentials across agents

2. Clear Access Policies Define explicitly:

• What information agents can read
• Which systems agents can modify
• Which actions require human approval

3. Human Approval for Consequential Actions

Low-risk routine action → Agent can proceed
High-risk or permission-changing action → Requires human approval

4. Supply Chain Verification

• Verify the source of tools, skills and dependencies
• Verify the integrity (has it been tampered with?)
• An agent is only as trustworthy as the tools it uses

5. Protected Audit Records When something goes wrong, you need to reconstruct:

• Which tool was called
• What authorization decision was made
• What the outcome was
• Where data was sent

Incident Response Readiness

Real-World Tool: NVIDIA OpenShell

OpenShell demonstrates these principles in practice:

• Enforces policies outside the agent's reach (the agent cannot override them)
• Sandboxed execution (limits what the agent can touch)
• Governs access to data, network and system resources

Built upon by partners:

• Cisco DefenseClaw → Adds governance layer
• JFrog → Scans and verifies agent skills before use

---

Step 4: Testing and Evidence Before Deployment

The Evidence Requirement

Saying "we think it's secure" is not acceptable. You need evidence that controls work.

What to Test

Test Category 1: Credential Boundaries
→ Can the agent obtain credentials beyond its scope? (It should not be able to)

Test Category 2: Data Exfiltration
→ Can sensitive data be sent to unauthorized destinations? (It should be blocked)

Test Category 3: Permission Escalation
→ Can the agent change its own permissions? (It should not be able to)

Test Category 4: Monitoring Integrity
→ Can the agent interfere with its own audit logs? (It should not be able to)

The Testing Lifecycle

1. Test before deployment
        ↓
2. Named owner reviews results
        ↓
3. Owner decides: ready or not ready
        ↓
4. Failed tests → corrective action (mandatory)
        ↓
5. Each failure becomes a repeatable test
        ↓
6. Re-test after any material change to model, tools or workflows
        ↓
7. Repeat cycle continuously

Key Principle: Every failure discovered becomes a permanent test case. This prevents the same vulnerability from reappearing in future releases.

Real-World Tools

---

Step 5: Choosing the Right Defensive Tools

Open vs. Closed Models for Defense

Why Open Models Matter During Incidents

During an active incident:

Closed model approach: Evidence may leave your environment
                       Limited ability to adapt investigation

Open model approach:  Keep sensitive evidence within your environment
                      Reproduce failures on your own systems
                      Test fixes before deploying them
                      Adapt investigation strategy as needed

How AI Supports Defenders

AI can assist security teams by:

• Finding vulnerabilities in code and systems
• Validating that fixes actually work
• Accelerating investigation and response

How to measure AI's value as a defensive tool:

• Reproducible findings (not one-time results)
• Verifiable fixes (proven to work)
• Measurable reduction in response time

Real-World Tools

---

Step 6: Collective Defense Through Open Sharing

Why Sharing Matters

Security knowledge shared across organizations creates a compounding defensive advantage.

Organization A discovers a failure
        ↓
Shares: what failed + which controls worked + how fix was verified
        ↓
Organizations B, C, D strengthen their own systems
        ↓
Collective security posture improves faster than attackers can adapt

What Should Be Shared

1. Evidence of what failed — specific, reproducible failures
2. Which controls worked — verified defensive measures
3. How fixes were verified — reproducible test methods

This is more valuable than general advice because it is specific, tested and actionable.

---

Summary: The Complete AI Security Framework

┌─────────────────────────────────────────────────────────┐
│                  AI SECURITY FRAMEWORK                   │
├─────────────────────────────────────────────────────────┤
│  PRINCIPLE 1: Engineering Discipline                     │
│  → Requirements + Controls + Owners + Evidence           │
├─────────────────────────────────────────────────────────┤
│  PRINCIPLE 2: Full Stack Responsibility                  │
│  → Model + Harness + Runtime all carry security duties   │
├─────────────────────────────────────────────────────────┤
│  PRINCIPLE 3: Environment Enforces Boundaries            │
│  → Controls hold even when agent reasoning fails         │
├─────────────────────────────────────────────────────────┤
│  PRINCIPLE 4: Evidence Before Deployment                 │
│  → Test, assign owner, fix failures, retest always       │
├─────────────────────────────────────────────────────────┤
│  PRINCIPLE 5: Right Tools for the Task                   │
│  → Open models for sensitive investigation               │
├─────────────────────────────────────────────────────────┤
│  PRINCIPLE 6: Collective Defense                         │
│  → Share findings, controls and verified fixes           │
└─────────────────────────────────────────────────────────┘

---

Self-Assessment Questions

1. Why is it insufficient to rely on an agent's reasoning to enforce security boundaries?
2. What four pieces of information should audit logs capture for every agent action?
3. An agent has permission to read customer records. A malicious document instructs it to email those records externally. List three controls that should prevent this.
4. What makes a security finding more valuable than general security advice?
5. Why might an open model be preferable to a closed model during an active security incident?