After studying this material, you should be able to:
- Describe how scientists currently use AI tools across different fields and research tasks
- Explain why AI usage patterns align with AI capability levels
- Analyze the relationship between task characteristics and AI suitability
- Evaluate scenarios for how AI could accelerate scientific discovery
- Assess the limitations and risks of AI-driven research
Step-by-Step Study Guide
Step 1: Understand the Research Setup
What Was Studied?
Researchers analyzed 207,049 Claude sessions from scientists using:
- Claude (general chat) — 128,914 sessions
- Claude Code (coding tool) — 78,135 sessions
- Claude Science (specialized scientific tool) — supplemental sample
How Was Privacy Protected?
A key methodological point:
Raw conversation → AI Classifier → Labels only (e.g., "chemistry," "data cleaning")
↓
Only labels are stored and analyzed
No research content is retained
Why Does This Matter?
This method allows large-scale measurement of AI use without exposing sensitive unpublished research.
Step 2: Learn How AI Use Varies by Scientific Field
The Core Finding
Field size alone does not explain Claude usage. Some fields use Claude far more than their size predicts.
Visual Mental Model
HIGH relative usage: LOW relative usage:
───────────────────── ─────────────────────
Computer Science Medicine/Clinical
Mathematics Nutrition
Statistics Veterinary Medicine
Economics Chemistry (lab-based)
Why the Difference?
| Factor | High-Use Fields | Low-Use Fields |
|---|---|---|
| Work type | Mostly digital/text/code | Physical lab work or patient contact |
| Tool fit | AI handles text/code well | AI cannot pipette or examine patients |
| Regulation | Fewer restrictions | Medical data regulations limit AI |
Claude Code Is Even More Concentrated
- Computer science = only 4.3% of world researchers
- But accounts for 32.7% of Claude Code sessions
- Reason: Claude Code is a coding tool → naturally fits computational fields
Key Takeaway
Computational subfields (computational biology, computational linguistics, computational neuroscience) are strong adopters regardless of their broader domain.
Step 3: Understand How AI Use Varies by Research Task
The Eight Task Categories
Scientists perform research through these task types:
- Data analysis
- Writing and communication
- Data gathering
- Reading and knowledge acquisition
- Theoretical analysis
- Design and development
- Ideation and hypothesis generation
- Other (administration, etc.)
The Usage Gap — A Critical Concept
This chart summarizes the mismatch between how much time researchers spend on tasks vs. how much they use Claude for them:
Task Category | Time Spent | Claude Use | Status
───────────────────────|────────────|────────────|──────────────
Data Analysis | 27.6% | 33.7% | OVERUSED ↑
Writing/Communication | 16.0% | 32.0% | OVERUSED ↑↑
Data Gathering | 28.6% | 11.5% | UNDERUSED ↓↓
Ideation/Hypothesis | 7.1% | 4.2% | UNDERUSED ↓
The Importance Paradox
Survey respondents ranked task importance for scientific discovery:
- 🥇 Ideation/Hypothesis Generation — most important (35.6 points)
- 🥈 Data Gathering (23.6 points)
- 🥉 Data Analysis (23.3 points)
- Writing (17.5 points)
The paradox: Scientists rank ideation as most important but use AI least for it.
Why the Mismatch?
| Task | Reason for Low AI Use |
|---|---|
| Data Gathering | Much of it is physical (collecting samples, clinical data) |
| Ideation | Scientists may prefer their own ideas; AI may genuinely be weaker here; hard to classify |
Step 4: Understand Claude Science as a Specialized Tool
What Makes Claude Science Different?
Claude Science connects to multiple research databases simultaneously, integrating fragmented workflows.
Example: A biologist studying drug-protein interactions normally needs separate databases for:
- Protein function
- Protein structure
- Compounds already tested
- Related drug trials
Claude Science combines these into one workflow.
How Claude Science Use Differs
| Metric | Claude | Claude Code | Claude Science |
|---|---|---|---|
| Human hours equivalent | 4.9 hrs | 5.9 hrs | 10.3 hrs |
| Doctoral-level content | 48% | 68% | 83% |
| High AI autonomy | 24% | 64% | 73% |
Key Insight
Specialized AI tools shift usage toward more complex, creative tasks. Ideation makes up ~8% of Claude Science sessions — roughly matching actual researcher time — compared to only ~4% on general Claude.
Step 5: Understand Theoretical AI Capability
How Was It Measured?
Researchers built a capability score for each research task using a tournament-style comparison:
Step A vs. Step B → Which can AI do better?
↓
Bradley-Terry Model (like chess Elo ratings)
↓
Each task gets a capability score
The Core Finding: Usage Tracks Capability
| Capability Quintile | Claude Use Ratio | Claude Code Use Ratio |
|---|---|---|
| Bottom 20% | ~0.2x | ~0.2x |
| Middle 20% | ~1.0x | ~1.0x |
| Top 20% | ~2.1x | ~3.9x |
Use ratio = share of Claude sessions ÷ share of researcher time
- Ratio of 1 = used exactly as much as time spent would predict
- Ratio of 2 = used twice as much as expected
What Predicts High AI Capability?
HIGH capability tasks: LOW capability tasks:
──────────────────────── ────────────────────────
✓ Cognitive work (+0.69) ✗ Physical work (−0.57)
✓ RL-suitable (+0.78) ✗ Interpersonal work (−0.31)
✓ Verifiable outcomes ✗ Regulatory restrictions (−0.38)
✓ Simulatable in computers
What Is RL Suitability?
Reinforcement Learning (RL) works best when:
- Success is easy to verify (did the code run? is the answer correct?)
- Many examples can be simulated in a computer
- Feedback is fast
This is why coding and math tasks have high AI capability — you can instantly check if code works or if a proof is valid.
Step 6: Understand Research Acceleration Scenarios
The Framework
AI accelerates research in two ways:
- Takes over tasks → frees researcher time → more output
- Generates better ideas → harder to measure, not modeled here
Three Scenarios
Scenario 1: Current Capabilities (Widely Adopted)
- AI takes on 27–52% of researcher time (median: 38%)
- Research output acceleration: 1.35x to 2.02x
Most accelerated: Statistics (2.02x), Computer Science (1.96x)
Least accelerated: Arts (1.35x), Anthropology (1.36x), Chemistry (1.41x)
Scenario 2: 75% of Non-Physical Steps Automated
- Computer Science: ~8x
- Statistics, Economics: ~6–7x
- Bench sciences (chemistry, biology): ~2–3x
Scenario 3: 100% of Non-Physical Steps Automated
- Formal/social sciences (no physical bottleneck): ~15–16x
- Bench sciences (physical bottleneck remains): ~3–4x
The Physical Bottleneck — A Key Concept
Field Type | Physical Work % | Max Acceleration
────────────────────|─────────────────|─────────────────
Computer Science | ~4% | ~15x
Statistics | ~4% | ~15x
Chemistry | 38–41% | ~3–4x
Biology | 38–41% | ~3–4x
Even perfect AI cannot run a physical experiment. Fields requiring lab work face a hard ceiling on acceleration.
The Cost Ceiling
Under current assumptions:
- 1 hour of AI agent = $30
- 1 hour of researcher = $95
- Maximum acceleration ≈ 16x (limited by AI cost)
If AI costs fall by half → acceleration ceiling doubles
Step 7: Understand Researcher Expectations
What Scientists Expect by 2030
| Task | % Expecting AI to Match Best Researchers |
|---|---|
| Data Analysis | 94% |
| Writing | 89% |
| Ideation | 72% |
| Data Gathering | 72% |
| Full research projects (no human) | ~40% |
| Major breakthrough (no human) | ~50% after 2035 |
Historical Accuracy of AI Forecasts
Forecasters have systematically underestimated AI progress.
Example: In 2022, experts predicted AI would achieve gold-level math olympiad performance by 2030–2035. It happened in 2025.
Exception: Physical-world tasks (like lab automation) have been harder to accelerate than predicted.
Step 8: Understand Limitations and Risks
Constraints on Acceleration
| Constraint | Explanation |
|---|---|
| Adoption lag | New technologies diffuse slowly; organizations must restructure |
| Physical bottlenecks | Lab work cannot be automated without robotics |
| Ideation bottleneck | If AI can't generate good questions, acceleration is limited |
| Quality vs. quantity | More papers ≠ better science; verification capacity matters |
| Organizational change | Research structure may need to fundamentally change |
Risks
| Risk | Description |
|---|---|
| Safety/alignment | Less human oversight → dangerous or unethical research |
| Biosecurity | AI that accelerates medicine could also produce dangerous knowledge |
| Researcher wellbeing | If AI takes rewarding tasks, job satisfaction may fall |
| Knowledge absorption | Humans may struggle to keep up with AI-generated findings |
Summary: The Big Picture
WHAT WE LEARNED:
1. Scientists use AI where it works best
(usage tracks capability scores strongly)
2. Formal/computational sciences adopt AI most
(cognitive, RL-suitable, digital work)
3. Physical and interpersonal tasks resist AI
(lab sciences face hard bottlenecks)
4. Specialized tools (Claude Science) enable
more complex, creative, autonomous work
5. Current AI could double output in some fields;
future AI could enable 15x in cognitive fields
6. The biggest gap: scientists want AI help with
ideation most, but use it there least
Self-Check Questions
- Why does computer science have disproportionately high Claude usage relative to its share of researchers?
- What is the "importance paradox" regarding ideation and hypothesis generation?
- What makes a research task highly suitable for reinforcement learning?
- Why do bench sciences face a hard ceiling on AI-driven acceleration even in optimistic scenarios?
- What does a "use ratio" of 2.0 mean, and which task categories achieve this?
- How does Claude Science differ from general Claude in terms of task complexity and autonomy?
- Why might researchers systematically underestimate AI progress?