Google opened Deep Review to the public on October 14, taking an AI manuscript assistant beyond conference pilots into broader technical-document checking. The consequential question is what happens after the system delivers its criticism: can a person turn those comments into verified improvements without spending more time investigating false alarms?
TENS Magazine’s analysis is that the useful unit of progress is a confirmed correction per hour of human checking. Counting documents processed, favorable reactions or issues flagged answers different questions. Reading Google’s launch announcement alongside the ICML pilot retrospective and the underlying research shows why those measures should remain separate.
A suggestion needs a disposition
Deep Review grew from Google’s Paper Assistant Tool, or PAT. Google says its conference programs reached more than 16,000 manuscripts. The public service initially accepts PDFs and plain text and offers two complimentary reviews weekly. Its architecture divides documents into sections, allocates reasoning resources, checks sections in parallel and consolidates the findings.
For a research group, the practical change is an additional opportunity to challenge a draft before submitting it. A useful trial would attach a disposition to every substantive comment: confirmed error, useful clarification, unsupported objection or unresolved question. Each category calls for a different response. A clearer paragraph may be valuable without correcting the science; a valid objection may require an experiment rather than an edit.
This classification also gives the author a stopping rule. Repeatedly revising until an AI report sounds favorable could optimize the document for that reviewer’s preferences. Recording why a comment was accepted or rejected preserves the reasoning behind the manuscript, including places where the author has stronger evidence than the tool.
The pilot measures activity, not a final verdict
The March ICML retrospective, written by Google researchers and conference program chairs, describes feedback on approximately 4,500 papers. Among survey respondents whose papers contained experimental results, 31% said the feedback led them to run new experiments. Among those with theory, 35.4% reported significant gaps that required more than an hour to fix. The feedback was private to authors and separate from formal peer review.
Those reports describe substantial work. They do not establish that every additional experiment improved a paper or that every flagged gap was independently confirmed. Participants chose to use the tool, and respondents chose to answer the survey. The findings therefore support testing an author-assistance workflow; they cannot, by themselves, establish a universal error rate or the amount of reviewer labor saved.
A stronger follow-up would compare original drafts with revised manuscripts, with assessors unaware of which received AI assistance. It should distinguish improvements in presentation from changes to conclusions and count new errors introduced during revision. Time spent rejecting incorrect advice belongs in the accounting alongside time spent repairing genuine problems.
A detection score has a boundary
The PAT research paper evaluates a selected portion of the SPOT benchmark: 26 mathematics and computer-science papers containing 29 equation or proof errors. It reports 89.7% detection for PAT against 55.2% for a single Gemini 3.1 Pro call. The researchers used their own grading procedure and warn that these results are not directly comparable with the original SPOT scores.
The reported difference is 34.5 percentage points, calculated by subtracting 55.2 from 89.7. That comparison concerns finding known errors in a small, selected collection. It does not mean that 89.7% of all criticisms issued on an ordinary manuscript will be correct. A deployment evaluation needs both the proportion of real errors found and the proportion of accusations that survive expert checking.
This matters especially when moving to a different document type. A tool that catches a contradiction in a proof may still miss an unsupported assumption in a design proposal. Teams could preserve a set of representative documents with adjudicated problems and repeat the assessment when the model or review pipeline changes. The relevant evidence would follow the actual version being used.
The document’s journey matters too
Google’s public-launch policy says uploaded documents are not used to train or improve its models and are stored for one year for display in the interface. The earlier NeurIPS author program described deletion within seven days after feedback delivery and completion of the program. These are different arrangements for different offerings.
For an organization evaluating the public service, that comparison creates a concrete intake question: is this particular draft approved for that storage period? Permission granted for a conference pilot should not silently become permission for a new service. The decision belongs before upload, alongside checking that the input contains the tables, figures and supporting context needed for a meaningful review.
Deep Review’s public release makes an extensive critique easier to obtain. Whether it makes verification easier is a further, measurable question. An author can answer it by retaining the original draft, the report, the evidence used to resolve each objection and the revised result. That record would show where the system helped, where it consumed effort and which conclusions still require independent scrutiny.
Illustrative photograph of computer code, not the Deep Review interface. Photo: Markus Spiske via Wikimedia Commons. CC0 1.0 Universal Public Domain Dedication. Center-cropped from 5,760 × 3,840 to 16:9 and resized to 2,400 × 1,350 pixels; no generative or content-altering edits.
