A more accurate AI answer is useful only if the user knows which questions the system has stopped answering. That distinction sits at the center of research highlighted by UC Riverside on September 23: internal model signals can help identify likely mistakes, creating an opportunity to withhold unreliable responses.
The university’s report describes work led by doctoral researcher Het Patel. The underlying paper was submitted in April; this week’s university account brings renewed attention to its implications. It should not be mistaken for a newly released consumer product or proof that AI systems can now recognize every error.
TENS Magazine’s analysis is that abstention deserves to be evaluated as a transfer of responsibility. An unanswered question moves somewhere: to a person, another system, or an unresolved queue. Measuring that destination alongside answer quality would make reliability claims more useful to anyone deciding whether to deploy an assistant.
Count the questions that leave the answer column
The study examines Llama-3.1-8B and Gemma-2-9B using multiple-choice tasks and tools called sparse autoencoders. Its central finding separates internal features associated with uncertainty from those associated with correctness. In one Llama experiment, three internal features supported a decision about whether to answer.
Table 5.5 reports 81.29% accuracy when the system answers 3,709 of 7,021 questions, or 52.8% coverage. The baseline answers every question at 61.91% accuracy. These figures describe different sets of answers; the higher percentage is not a claim that the system correctly completes 81.29% of all incoming work.
TENS Magazine’s calculation puts both results on the same denominator. Multiplying 52.8% coverage by 81.29% accuracy yields approximately 42.9 correct answers per 100 submitted questions, about 9.9 incorrect answers and 47.2 abstentions. The baseline yields approximately 61.9 correct answers and 38.1 incorrect answers per 100. Rounding means these are illustrative rates, not reconstructed integer counts.
That accounting explains why a smaller number of correct automatic answers can still be desirable. A user may prefer an explicit pause to an incorrect response. But the choice is incomplete until the rejected work has an owner. A system that declines difficult questions without providing a useful next step can improve its reported answer accuracy while leaving the overall service less helpful.
An established tradeoff with a different signal
There is a longer research history behind this distinction. Yonatan Geifman and Ran El-Yaniv’s 2017 paper, Selective Classification for Deep Neural Networks, studied systems that reject some inputs to control prediction risk. Its experiments concerned image classification, rather than today’s conversational assistants.
Reading the two studies together changes the question an evaluator should ask. The useful comparison is not simply whether an assistant becomes more accurate after it declines work. It is whether its method of choosing what to decline improves the balance between mistakes and unanswered requests, compared with another selection method serving the same workload.
A fair comparison could fix coverage first. Give two systems the same questions and require each to answer the same proportion, then compare their error rates. Alternatively, fix an acceptable error rate and compare how much work each can complete. Those are proposed evaluation choices, not additional results from either paper.
Make the handoff part of the test
For a practical trial, TENS Magazine proposes tracking three outcomes together: correctly answered requests, incorrect answers that reach users, and declined requests that eventually receive a satisfactory resolution. Each category should retain its original denominator. Moving a request into a review queue should not make it disappear from the performance report.
Review capacity then becomes part of the deployment decision. If a proposed assistant declines roughly half its workload, the receiving team needs to know whether those questions take seconds or hours to resolve. A trial could record waiting time and reviewer effort without assuming that every handoff is either a failure or a success.
The review process also needs a way to expose unnecessary refusals. Some rejected questions may have been answerable correctly. Sampling them would help distinguish prudent restraint from a system that avoids work indiscriminately. Keeping that sample separate from the material used to choose the abstention threshold would make the resulting assessment more informative.
The present evidence has clear boundaries. The researchers’ tests concern multiple-choice questions and two models of similar scale. They do not establish the same performance for open-ended writing, other model sizes or a real customer-support queue. Those differences need testing before an operator treats the reported rates as a staffing forecast.
The immediate product opportunity is a more accountable pause. An assistant could explain that it is withholding an answer, preserve the question and identify the next review step. Whether that design helps depends on what happens after the pause. The strongest reliability report would show the answer, the abstention and the eventual resolution together.
Illustrative software photograph; not an image of the research models. Photo: Markus Spiske via Wikimedia Commons. CC0 1.0 Universal Public Domain Dedication. Center-cropped from 5,760 × 3,840 pixels to 16:9 and resized to 2,400 × 1,350 pixels; no generative or content-altering edits.