Skip to content
AI

AI’s 32-Minute Biology Run Leaves Two More Clocks to Measure

TENS examines how distributed AI research should account for computation time, expert review and the experimental decisions that follow.

Share Email
Colorful software code displayed on a computer monitor
Source code displayed on a computer monitor. Photo: Markus Spiske via Wikimedia Commons. CC0 1.0 Universal Public Domain Dedication. Center-cropped from 5,760 × 3,840 pixels to 16:9 and resized to 2,400 × 1,350 pixels; no generative or content-altering edits.

Thirty-two minutes is a striking time for an AI analysis run. It is a less useful measure of scientific progress until readers know where the clock starts and stops. A new account of distributed agents exploring biological data makes that distinction unusually concrete: generating research leads, checking their evidential basis and testing them experimentally are separate jobs.

On October 21, OpenAI described work with PerturbAI and Scripps Research in which thousands of GPT-Rosalind agents interpreted existing data from millions of mouse brain cells. The company reports that the analysis took 32 minutes. Agents connected gene-disruption results with scientific literature to produce possible explanations and research leads.

TENS Magazine’s analysis is that the most useful advance here may be a more inspectable route from a large dataset to a shortlist of experiments. Judging that route requires three separate clocks: computation, expert checking and experimental follow-up. Collapsing them into one speed claim would obscure where researchers still have work to do.

What the stopwatch leaves outside

OpenAI says blinded experts preferred distributed-agent reports over single-agent reports 75 percent of the time. It also describes a comparison holding the model and resources fixed in which the distributed approach searched more literature. Those are reported evaluations of interpretation, rather than evidence that each proposed mechanism has been established experimentally.

For a laboratory deciding whether to use this approach, TENS would begin with the handoff. What exactly arrives after the timed run: a readable report, a checked finding or an experiment ready to execute? Each has value, but they replace different amounts of human work. A report that arrives quickly yet takes considerable checking could still broaden the search while saving less staff time than its generation speed suggests.

Our proposed accounting would record elapsed machine time alongside the expert minutes needed to accept, amend or reject each shortlisted lead. It would also retain rejected leads and the reasons for rejection. That makes the denominator visible: a handful of compelling examples cannot establish how much useful work the entire run delivered.

An accessible report is not a reproduced result

The researchers’ public Distributed Agents repository provides a useful distinction. Its hosted interface allows readers to explore findings, claims and supporting evidence, but does not include the raw dataset files needed for operations that recompute statistics. The local repository supplies a route to data and manuscript analyses. Its documentation says reproducing saved analyses requires no model calls or API credentials.

These are different levels of access. Browsing an explanation lets a reader inspect an argument; rebuilding a saved analysis checks whether the released inputs and methods produce the recorded result. Running agents again asks another question: whether a fresh execution reaches a similarly defensible interpretation. Completing one level should not silently earn credit for the others.

TENS proposes tracking one candidate explanation across those boundaries. Identify the underlying measurement, the cited evidence supporting its interpretation and the precise new claim added by the agent. Then record whether a reviewer could check each component using the hosted interface alone or needed the local data. This would expose practical verification costs that a polished narrative can conceal.

The repository identifies the associated manuscript as a preprint. TENS has not independently rerun its analyses, and the public software release should not be read as independent replication. Its importance for this assessment is narrower: it gives other researchers an explicit starting point for examining the work.

The next experiment is the decisive handoff

A useful follow-up would compare how teams choose experiments with and without agent assistance under the same review-time budget. Before seeing outcomes, reviewers could record which leads they select, why they consider them testable and what result would count against the explanation. That would help distinguish a persuasive report from a proposal that genuinely improves an experimental decision.

Negative outcomes belong in that record. If an agent directs attention toward an initially plausible mechanism that fails a well-designed test, the result can still clarify the biology. But evaluation should distinguish that learning from avoidable failures caused by unsupported citations, misunderstood measurements or an explanation that was never specific enough to test.

The wider measurement problem is also on the agenda at Microsoft Research and Carnegie Mellon University’s October 22–23 workshop on AI agent evaluation. The organizers plan public reports covering open problems, areas of agreement and disagreement, and gaps in current methods. That meeting is a separate initiative, not a validation of the biological study.

The connection is practical: evaluation must follow the work an agent is supposed to improve. Here, the strongest next evidence would connect generated interpretations to documented review effort and subsequent experimental choices. The 32-minute result is an intriguing beginning. Its lasting significance will depend on what researchers can verify and do with the output after that clock stops.

Illustrative photograph of software code, not the study’s software or biological data. Photo: Markus Spiske via Wikimedia Commons. CC0 1.0 Universal Public Domain Dedication. Center-cropped from 5,760 × 3,840 to 16:9 and resized to 2,400 × 1,350 pixels; no generative or content-altering edits.