After studying this material, you should be able to:

1. Explain the three-step framework of scientific modeling (build, constrain, interpret)
2. Understand why a single best-fit model is insufficient for scientific insight
3. Describe how AI tools (specifically SBI and LLM-based agents) assist each modeling step
4. Articulate why distributions of models matter more than single solutions
5. Evaluate the limits of AI in replacing human scientific understanding

---

Step-by-Step Teaching

Step 1: Why Scientific Models Exist

Before understanding AI's role, you need to understand what a scientific model actually does.

Core idea: A model is a simplified mathematical description of a real system that lets you make predictions.

The Hodgkin-Huxley Example

In 1951, two scientists wanted to understand how nerve cells communicate using electrical pulses called action potentials.

They built a set of equations describing the nerve cell membrane as an electrical circuit.

Key moment: To make their equations match real measurements, they had to raise one variable to the fourth power (multiply it by itself three times).

They didn't fully know why the fourth power worked. They just knew simpler versions didn't fit the data as well.

Decades later: Scientists discovered the relevant protein had exactly four identical pieces arranged around a central pore.

The model, built purely to fit data in 1951, had accidentally predicted a physical structure nobody had seen yet.

This illustrates the true power of a good model:

• It reproduces existing data ✓
• It predicts things it was never designed to predict ✓
• It generates understanding, not just numbers ✓

---

Step 2: The Three-Step Framework of Scientific Modeling

All scientific modeling follows this repeating cycle:

BUILD → CONSTRAIN → INTERPRET → (revise and repeat)

Step 2a: Building the Model

What it means: Deciding the structure of your equations — what variables exist and how they relate.

The challenge: The space of possible model structures is essentially infinite. Scientists historically relied on expertise and intuition to narrow it down.

---

Step 2b: Constraining the Model

What it means: Tuning the adjustable numbers (parameters) inside your chosen model structure to match real data.

The restaurant bill analogy:

Imagine a group dinner totaled $120, but you don't know who ordered what. Many different combinations of orders could produce that same total.

This is exactly how model parameters work:

• Different combinations of parameter values can produce identical-looking outputs
• Picking just one combination hides all the other valid ones
• Other valid combinations might make different predictions about untested situations

The problem with single best-fit values:

Single fit → "Here is ONE answer that works"
             (hides all other equally valid answers)

The solution → Bayesian Inference:

Instead of finding one best set of parameters, find the entire distribution of parameter sets that fit reasonably well.

Bayesian approach → "Here is the RANGE of answers that work,
                     weighted by how well each fits"

This tells you:

• Which parameters are tightly constrained by data (narrow distribution)
• Which parameters remain uncertain (wide distribution)
• Where you need more experiments

The practical problem: Standard Bayesian inference requires calculating how likely your measurements are under every possible parameter combination. For complex models, this is computationally impossible.

---

Step 2c: Interpreting the Model

What it means: Asking what the model tells you about the real physical system.

Hodgkin and Huxley's interpretation:

"The fourth power might mean four particles must move into place before the gate opens"

Critical insight: This interpretation only had weight because they had ruled out simpler alternatives. If a third-power model had worked equally well, the four-particle hypothesis would have been unjustified.

Understanding comes from knowing what didn't work, not just what did.

---

Step 3: How AI Assists — Constraining Models with SBI

The problem AI solves: Standard Bayesian inference is often computationally impossible for complex models.

The solution: Simulation-Based Inference (SBI)

Here's how it works, step by step:

Step 1: Pick random parameter values
Step 2: Run the model with those parameters → get output
Step 3: Record the (parameters → output) pair
Step 4: Repeat 100,000 to millions of times
Step 5: Train a neural network on all these pairs
Step 6: Given NEW real data, the network returns the 
        distribution of parameters that could produce it

Key requirement: You only need a simulator — something where you put parameters in and get data out. You never need a formula for working backwards.

Why this matters:

• Many real-world models can only be run forward (parameters → data), never reversed
• Examples: networks of neurons, colliding galaxies, climate systems
• SBI works for all of them

Real-world applications of SBI:

---

Step 4: How AI Assists — Building Models

The traditional approach: Scientists propose a handful of candidate model structures from expertise, test them, revise, repeat.

The AI approach: LLM-based agents now automate this loop:

1. Given data + natural language description of system
2. LLM writes code for candidate model(s)
3. System constrains parameters, compares to real data
4. LLM revises model structure
5. Loop repeats until model fits data accurately

The same fundamental problem reappears — one level up:

Just as many parameter sets can fit one model, many different model structures can fit the same data equally well.

Run the agent twice on same data
→ May get two DIFFERENT models
→ Both equally valid
→ Returning only one hides the other

The better approach: Treat model-building like parameter-constraining — seek a distribution of plausible models:

Many candidate models proposed
→ Each weighted by fit to data
  (models needing excessive flexibility get less credit)
→ Weak candidates eliminated
→ Variations on strong candidates proposed
→ Result: distribution of plausible model structures

What a distribution of models tells you:

---

Step 5: How AI Assists — Interpreting Models (The Hard Part)

Why interpretation is hardest to automate:

• Model fit can be measured objectively
• "Understanding" cannot be easily measured
• It's unclear how to tell an AI to maximize insight

Two proposed approaches:

Approach 1 — Test predictions under disturbance:

Does the model correctly predict what happens when you interfere with the system? (e.g., blocking some sodium channels in a nerve cell)

Approach 2 — Measure impact on human understanding:

Do the AI's outputs help human scientists understand better?

Example: An AI system for quantum experiments was trained to write a short program generating a whole family of related experiments, so a physicist could read the underlying design principle directly.

Why distributions remain essential for interpretation:

Single model → Easy to read meaning into parts 
               NOT actually constrained by data
               (false confidence)

Distribution of models → Shows which features are 
                         genuinely required by data
                         vs. which are arbitrary choices

---

Step 6: The Bigger Picture — Where This Is Heading

Scale Changes Everything

As AI handles more of the modeling work, the size limit on models lifts. We can model systems too complex to write by hand.

The fundamental shift in scientific questions:

This Already Happens in Other Fields

• Climate science: Many simulations with different assumptions; results reported as a spread
• Particle physics: Maps of which theory versions are ruled out, not just yes/no answers

---

Step 7: The Core Tension — AI Speed vs. Human Insight

What AI can now do in an afternoon (vs. Huxley's three weeks):

• Find equations fitting the data
• Potentially fit data more closely than humans did

What AI must still deliver to provide genuine understanding:

Not just: "Here is the model that fits"

But also:
✓ Which parts are pinned down by data
✓ Which parts remain open/uncertain  
✓ Which alternatives failed and why
✓ What new puzzles were uncovered

"The purpose of computing is insight, not numbers." — Richard Hamming, 1962

The process of building a model is where much of the understanding comes from. Hodgkin and Huxley understood their model deeply because they had built it themselves and knew what hadn't worked.

AI must replicate not just the output of that process, but the epistemic trail — the record of what was tried, what failed, and what remains uncertain.

---

Summary Table

The unifying principle across all three steps:

A single best answer — whether a parameter value, a model structure, or an interpretation — hides the uncertainty that is essential for genuine scientific understanding.