Artificial intelligence systems are increasingly expected to reason with material they did not generate: search results, retrieved documents, user notes, answer keys, workplace rules, and data supplied by other tools. A new study argues that the usual test of whether a model can resist bad context is incomplete. A system that ignores every outside signal may look robust while failing the reason it was given context in the first place.
Researchers from Duke University, the National University of Singapore, the University of California campuses at Berkeley and Irvine, Northeastern University, and Nanyang Technological University introduced a benchmark called MIST and a training method called SCOPE. Their paper was submitted to arXiv on August 6.
The work tests a narrower but important question: can a model reject a plausible wrong hint, benefit from a correct hint, and remain steady when extra information is irrelevant? The authors call that ability selective trust.
One question, four information environments
MIST contains 1,000 reasoning items rendered in four matched versions, creating 4,000 evaluation rows. One version gives the question without added context. A second adds information pointing toward a plausible wrong answer. A third supplies context supporting the correct answer, while a fourth adds related but non-answer-bearing material.
Because the question, answer choices, and correct answer remain fixed, the design isolates what the added signal changes. Eight hundred items were adapted from established reasoning and question-answering datasets; 200 were written as longer scenarios by human annotators. The subject mix includes mathematics, science, general knowledge, consumer finance, and workplace policy.
Across 23 proprietary and open-weight models, the researchers report that misleading context reduced accuracy by an average of 17.1 percentage points. Their paired metric, called SC2W, counts cases in which a model answers correctly without extra context and switches to a wrong answer after receiving a misleading signal. That distinction avoids blaming the context for questions a model could not solve in the first place.
The pattern varied substantially by model. GPT-5.5 recorded a 10.5 percent correct-to-wrong flip rate, while Claude Opus 4.7 recorded 24.6 percent. Among the tested open-weight systems, results also ranged widely. These are controlled benchmark rates, not estimates of how often deployed assistants fail in daily use.
TENS Analysis: distrust is not the same as judgment
The study exposes a blind spot in one-dimensional safety scores. If an evaluation contains only wrong hints, a model earns credit every time it refuses to be influenced. That score cannot reveal whether the system evaluated the evidence or merely learned to discount anything placed around the question. A useful assistant needs a two-sided test: resistance when context is wrong and responsiveness when it is right.
This matters beyond chat. Retrieval systems, research agents, and workplace assistants operate inside pipelines where external information is the point of the product. Training them to treat context as inherently suspect could reduce one class of error while quietly destroying the value of search, private databases, or verified tool output. Selective trust reframes reliability as an evidence-routing problem rather than a simple obedience problem.
The benchmark also suggests a practical evaluation rule. Product teams should not report a single accuracy number for context-enabled AI. They should measure the same tasks with no context, helpful context, misleading context, and irrelevant context, then inspect how individual answers move. Aggregate performance can hide a system that gains robustness by becoming less receptive to good information.
What SCOPE changes
The proposed training method mines examples a base model gets right alone but gets wrong after a misleading hint. It then balances those resistance examples with matched clean, correct-context, and irrelevant-context cases during preference optimization.
On Qwen3-4B, SCOPE reduced the paired flip rate from 35.0 percent to 16.3 percent while slightly improving clean accuracy and preserving performance in the two control conditions. On Llama-3.2-3B, the flip rate fell from 31.5 percent to 20.6 percent. A comparison method trained only against misleading context raised resistance but sharply reduced correct-context accuracy on the Llama model, illustrating the tradeoff the four-condition design was built to detect.
Earlier peer-reviewed research provides useful context. An International Conference on Machine Learning study showed that irrelevant sentences can distract language models on mathematical problems. A NeurIPS paper found that biasing cues can alter answers without being acknowledged in the model’s written reasoning. MIST combines those concerns with a control that asks whether useful evidence is still being used.
What remains unproven
The new paper is a preprint, and its mitigation experiments cover only two trainable model families. MIST is text-only, and most of its items originate in public benchmarks, so prior exposure cannot be ruled out. The authors also caution that performance on controlled prompts does not establish that a model’s chain of thought faithfully describes how it reached an answer.
The central contribution is therefore a measurement framework, not proof that selective trust has been solved. The next step is to test the idea in live retrieval and agent systems, where sources conflict, permissions differ, documents change, and apparently authoritative context may itself be stale. Reliability will depend not only on whether an AI can resist a bad hint, but on whether it can recognize when outside evidence deserves to change its mind.
Sources: Learning When to Trust via Selective Context Preference Optimization research paper and official released benchmark artifacts; Large Language Models Can Be Easily Distracted by Irrelevant Context, published at the International Conference on Machine Learning; Language Models Don’t Always Say What They Think, published at NeurIPS.
Featured image: Computer workstations in a university library. Photo: Mshuang2 via Wikimedia Commons. CC0 1.0 public domain dedication. Center-cropped from the original to 16:9 and resized to 1,920 × 1,080 pixels; no generative or substantive alteration.
