How AI Models Find Vulnerabilities—and What It Costs

Concept 1: The Core Problem — Why Finding Vulnerabilities First Matters

The fundamental idea: Security is a race between attackers and defenders.

Think of your codebase like a house:

• Attackers try to find unlocked windows from the outside
• Defenders know the floor plan and can check every window from the inside

The article opens with a real example:

OpenAI tested two models in a sandbox → models found a vulnerability → accessed the internet → reached Hugging Face's production database

Key insight: AI makes attackers more capable. But defenders get the same tools plus one critical advantage:

Defender Advantage = AI Tools + Full Knowledge of Own Codebase

Attackers must probe blindly. Defenders can read the source directly.

The strategic conclusion: Find your vulnerabilities before attackers do.

---

Concept 2: What a Benchmark Is and Why It's Needed

A benchmark is a standardized test that measures performance consistently across different subjects — here, different AI models.

Why is a benchmark necessary?

Without one, you'd have to answer questions like:

• "Is Model A actually better than Model B at finding bugs?"
• "Is paying 5x more worth it?"
• "How long will a scan take?"

...through expensive, inconsistent trial and error.

DeepsecBench solves this by creating a controlled, repeatable test:

Same codebase → Same vulnerabilities → Run each model → Compare results

Think of it like standardized testing for AI security scanners.

---

Concept 3: How the Benchmark Is Constructed

The test setup:

Why use a historical commit? Because the "correct answers" (the vulnerabilities) are already known — they were fixed in the next commit. This gives a ground truth to measure against.

Why keep the benchmark secret?

"We don't disclose the repository, the commit, the files, or the findings"

If models knew the answers, they could memorize them. The article notes:

"A model reciting memorized fixes would score near-total recall. Instead, the best run finds 30.7%"

This proves the test is genuinely hard and not gameable.

---

Concept 4: The Scoring Metrics — Recall, Precision, and F2

This is the most technical concept. Let's build it step by step.

Step 4a: Recall

Recall = "Of all real vulnerabilities, how many did the model find?"

Recall = Found Real Vulnerabilities / Total Real Vulnerabilities

Example: Found 100 of 231 known bugs → Recall = 43%

Low recall is dangerous → missed vulnerabilities stay in your code, unfixed.

---

Step 4b: Precision

Precision = "Of everything the model flagged, how many were actually real?"

Precision = True Findings / (True Findings + False Alarms)

Example: Model flags 150 things, 100 are real → Precision = 67%

Low precision is annoying but not dangerous → developers waste time on false alarms, but your code isn't less secure.

---

Step 4c: Why Recall Matters More Here

The article makes a deliberate value judgment:

Therefore: Recall should be weighted more heavily than precision.

---

Step 4d: The F2 Score Formula

The benchmark combines both into one number using F2 score:

Score = 100 × (5 × P × R) / (4P + R)

The "2" in F2 means recall is weighted twice as much as precision.

Compare to F1 (equal weight) vs F2 (recall-heavy):

• F1: balanced
• F2: penalizes missed vulnerabilities more severely ✓

Example interpretation:

• Top score in the benchmark: ~35.58
• This is intentionally hard — even the best model misses most vulnerabilities

---

Concept 5: The Cost-Performance Tradeoff

Now that you understand scoring, the benchmark reveals a practical insight:

Higher price no longer buys proportionally more performance.

The key pattern:

Diminishing returns: 2x the cost ≠ 2x the security

For a full production codebase (~100x the benchmark size):

• Kimi K3 sweep ≈ $1,200
• Top frontier model ≈ $5,000+

---

Concept 6: Building a Multi-Model Scanning Strategy

Since no single model is perfect for every situation, the article introduces a layered scanning approach.

The variables to balance:

Model Power ←→ Cost ←→ Speed ←→ Frequency

Practical strategies:

For a Startup:

Every merge    → Grok 4.5 (cheap, fast)
Milestone release → Frontier model (thorough, expensive)

For a Large Enterprise:

Critical services     → Frontier model, full reasoning
Key pull requests     → Same model, lower reasoning setting
Rest of codebase      → Continuous Kimi/Grok sweep

Reasoning settings matter too:

GPT-5.6 Sol example:

---

Concept 7: Infrastructure — Handling Spiky AI Workloads

The technical challenge:

Security scans are not steady workloads. They:

• Consume massive tokens in short bursts
• Hit rate limits from individual providers
• Require multiple different model providers

The solution — AI Gateway:

Your Scanner → [Single AI Gateway Endpoint] → Any Model Provider

Benefits:

• One API key covers all models
• Automatic routing, retries, and failover
• No per-provider rate limit management

In practice:

pnpm deepsec process --project-id my-app --agent pi --model xai/grok-4.5

One command, one key, any model on the leaderboard.

---

Summary: The Full Mental Model

PROBLEM:
  Attackers use AI → more dangerous than ever

SOLUTION:
  Defenders use AI first → scan your own code before attackers probe it

HOW TO MEASURE:
  DeepsecBench → Recall + Precision → F2 Score (recall-weighted)

HOW TO CHOOSE:
  Match model power + cost + speed to scan frequency and codebase criticality

HOW TO RUN:
  AI Gateway → single endpoint → burst capacity → any model

The core principle the article keeps returning to:

Defenders see the whole system. Attackers probe blindly.
The same AI model is more powerful in your hands — but only if you use it first.