← Home

AutoDiscovery in AstaLabs automates scientific discovery by generating hypotheses from data.

Peter Bubenik · AllenAI (Olmo model) · · Source
AutoDiscovery in AstaLabs automates scientific discovery by generating hypotheses from data.

TL;DR

AutoDiscovery flips the usual “AI for science” script by starting from your dataset (not your question) and automatically generating, testing, and ranking new hypotheses—so hidden discoveries can surface in hours instead of months.

Key Discoveries (12 total)

  1. AutoDiscovery is a “data-first” scientific discovery system that generates its own research questions from a structured dataset instead of waiting for a human-provided question.

  2. It runs an end-to-end discovery loop automatically: writes hypotheses in plain language, proposes experiment plans, generates Python code, runs analyses, interprets statistical results, and then uses those results to propose the next hypotheses.

  3. It outputs a reproducible list of novel research directions, meaning each suggested direction comes with the code and analysis trail needed to re-run and verify it.

  4. Researchers are already using it to find unexpected patterns across fields, including marine ecosystem relationships and cancer mutation patterns.

  5. Some AutoDiscovery-generated social science findings were strong enough to be published in a peer-reviewed paper after independent verification.

  6. AutoDiscovery addresses a common failure mode in open-ended AI exploration (wandering randomly or inheriting training-data bias) by using a principled method to decide what to test next.

  7. Its core “what to pursue next” signal is Bayesian surprise, a measure of how much the system’s belief changes after seeing evidence (in simple terms: how much the data forces it to update its mind).

  8. It explicitly tracks both the size and direction of belief change, so it can value results that strengthen a hypothesis and also results that strongly disconfirm it (because contradictions can be especially informative).

  9. It uses Monte Carlo Tree Search (MCTS) to explore efficiently, balancing “try new ideas” with “dig deeper where results look promising” in a huge space of possible hypotheses.

  10. In AstaLabs, AutoDiscovery makes discovery progress inspectable in real time, showing a table of tested hypotheses and a growing search tree of how ideas branch and evolve.

  11. It provides transparent auditing via an inspector view, where users can see the full hypothesis, the statistical analysis, and the exact Python code used—so results aren’t a black box.

  12. A concrete genomics finding: a potential mutual-exclusivity pattern in breast cancer mutations, where among patients with PIK3CA mutations, TP53 mutations appeared less frequent than expected by chance; the system’s belief mean shifted from 0.50 to 0.82, which is why it flagged the result as highly “surprising” and worth follow-up.

Why This Matters

Most scientific tools—even advanced AI systems—still depend on humans to ask the right question first. AutoDiscovery targets the bottleneck scientists often struggle with: figuring out what’s worth investigating at all. By automatically proposing and testing many plausible hypotheses, it can turn a dataset from a static spreadsheet into an “idea generator” that surfaces leads people might never think to ask about.

Practically, this can compress weeks or months of exploratory analysis into a much shorter cycle, while keeping the work verifiable (you can inspect the code and rerun the analysis). And because it rewards “belief-changing” results—including surprising contradictions—it’s designed to hunt for genuinely informative patterns, not just obvious correlations.

The Research Approach

AutoDiscovery is built around a simple idea: instead of a human choosing one question to test, the system treats the dataset like a landscape to explore and it generates many testable questions on its own.

Here is the step-by-step loop it runs, end to end:

  1. Start with a structured dataset and a bit of context You give AutoDiscovery a dataset (CSV, JSON, Parquet) and a short description of what the data represents. That context is used to “seed” the system’s starting expectations, so it does not begin completely blind.

  2. Generate candidate hypotheses in plain language It proposes hypotheses as sentences a researcher would recognize, such as “In group A, variable X is less common than expected” or “Feature Y is associated with outcome Z.”
    A hypothesis here means a specific claim that can be checked against the dataset.

  3. Create an experiment plan for each hypothesis For every hypothesis, it outlines how to test it: what subset of data to use, what comparison to run, and what statistical check is appropriate.
    An experiment plan here means a short recipe for turning a question into a concrete analysis.

  4. Write Python code to run the test AutoDiscovery then generates the actual analysis code and executes it. This is important because it forces the system to “commit” to a precise test rather than staying at the level of vague suggestions.

  5. Interpret the results and update its belief Before running the test, AutoDiscovery has a prior belief about whether the hypothesis will be supported. After seeing the data, it forms a posterior belief.
    Prior/posterior are just “before” and “after” confidence levels, represented as probabilities.

  6. Score the result using Bayesian surprise It measures how much its belief changed after seeing evidence. This is Bayesian surprise: a numeric way to capture “how much the data forced me to update my mind.”
    Crucially, it tracks both:

  • Magnitude: big change = informative result
  • Direction: did the evidence strengthen the hypothesis or weaken it?
  1. Choose what to test next using Monte Carlo Tree Search (MCTS) AutoDiscovery is not just running random tests. It uses Monte Carlo Tree Search (MCTS) to decide where to spend its next experiments.
    MCTS is a search strategy that grows a branching “tree” of possibilities and balances:
  • exploring new branches (new kinds of hypotheses)
  • digging deeper where early results look promising
  1. Output reproducible research directions At the end, you get a ranked set of hypotheses with the full trail: the hypothesis text, the plan, the statistics, and the Python code—so a human can rerun, audit, and extend the work.

A helpful analogy: think of AutoDiscovery as a lab assistant that can run hundreds of small, well-documented pilot studies overnight, and it prioritizes the pilots that most changed its expectations.

How It Works in Practice

The article gives several concrete “in the wild” examples of the same workflow.

Cancer genomics case study (breast cancer mutations)

  • The team explored a dataset of breast cancer mutations with oncologists.
  • AutoDiscovery began broadly by searching for mutation co-occurrence patterns (which mutations tend to appear together or not together).
  • It then narrowed into more specific hypotheses along a branch of its search tree.
  • One hypothesis it tested: among patients with PIK3CA mutations, TP53 mutations appear less frequent than expected by chance.
  • This pattern is called mutual exclusivity, meaning two mutations rarely show up together. In simple terms: if mutation A is present, mutation B is unusually absent. Researchers care because it can hint that the two mutations do similar jobs in the cell, or that having both is harmful to the cancer cell.
  • AutoDiscovery’s belief update made it stand out:
    • Prior mean belief: 0.50 (neutral, basically “could go either way”)
    • Posterior mean belief: 0.82 (much more confident it is real in this dataset)
  • Because that shift was large, the system marked it as highly surprising and then proposed follow-up hypotheses to validate and interpret it.

Marine ecosystem case (20 years of data) The article notes researchers used AutoDiscovery to uncover trophic relationships in long-running marine ecosystem data.
A trophic relationship means “who eats whom” or how energy moves through an ecosystem (for example, changes in one species’ population aligning with changes in another species up or down the food chain). The key methodological point is that AutoDiscovery can scan many plausible relationships across many variables and years without a human pre-selecting which relationships to test.

Social science case (publishable findings) The article reports that some AutoDiscovery-generated findings in social science were strong enough to be published after independent verification. Methodologically, this highlights the intended workflow: AutoDiscovery proposes and tests; humans then verify, interpret, and decide what is publication-worthy.

AstaLabs interface as a “method in action” In AstaLabs, you can literally watch the method operate:

  • A table fills with tested hypotheses.
  • Each row shows Before → After belief and a Surprisal score.
  • A growing tree visualization shows how one idea led to the next.
  • An inspector panel lets you open any result and see the exact code and statistical output.

What Makes This Different

1. It is “data-first,” not “question-first” Traditional statistics and most AI research assistants start with: “What is your question?”
AutoDiscovery starts with: “What does this dataset contain that is worth asking about?”
That is a major shift because, in real research, choosing the question is often the bottleneck.

2. It has a principled way to avoid aimless exploration Open-ended AI exploration often fails in two ways the article calls out:

  • it wanders randomly
  • it repeats biases from training data

AutoDiscovery’s novelty is using Bayesian surprise as the steering wheel. Instead of chasing whatever sounds plausible, it chases what the data makes it reconsider.

3. It values disconfirming evidence, not just “positive results” Because it tracks the direction of belief change, a strong negative update can be treated as valuable.
In plain terms: if the system expected something to be true and the data strongly says it is not, that contradiction is treated as informative rather than discarded.

4. It scales exploration with MCTS rather than brute force The space of possible hypotheses is extremely large. MCTS gives a practical way to allocate limited experiment budget across many branches, similar to how a good investigator does not follow every lead equally.

5. It produces auditable, reproducible artifacts Many AI tools produce suggestions that are hard to verify. Here, each result comes with the analysis code and statistical details, so the output is closer to a lab notebook than a chat response.

Overcoming Challenges

Challenge 1: Open-ended systems can drift or mirror training bias

  • Problem: Without a rule for “what next,” the system can test arbitrary ideas or default to what it has seen in training text.
  • Solution: Use Bayesian surprise to prioritize hypotheses that are most information-rich given the actual dataset, not just the model’s preferences.

Challenge 2: The hypothesis space is effectively infinite

  • Problem: You cannot test everything; even a simple dataset can generate countless possible relationships.
  • Solution: Use Monte Carlo Tree Search (MCTS) to balance breadth (new ideas) and depth (follow promising threads), so compute is spent where it is most likely to pay off.

Challenge 3: Trust and verification

  • Problem: Scientists need to know exactly how a result was produced.
  • Solution: AstaLabs provides an Inspector view with the full hypothesis, the statistical analysis, and the exact Python code. This makes the system’s work reproducible and reviewable.

Challenge 4: Compute cost and practical usability

  • Problem: Runs are compute-intensive and can take hours.
  • Solution: The platform uses an experiment budget (“hypothesis credits”) so users can start small (for example, fewer than 10 hypotheses) and scale up once they see value, with a cap (up to 500 hypotheses) to keep runs bounded.

Challenge 5: Data handling and privacy logistics

  • Problem: Users may worry about uploading sensitive datasets.
  • Solution: The workflow includes a confirmation that data is not confidential, datasets are deleted after 7 days, and the system retains only the outputs needed to reproduce results (hypotheses, plans, code, results).

More to study