A new AI evaluation pilot tries to solve a basic trust problem: the company being tested should not see the exam, and the examiner should not receive the company’s proprietary model. Google DeepMind, MLCommons, OpenMined, AVERI, and the Singapore AI Safety Institute used protected computing infrastructure to keep both sides’ assets confidential while a closed model was evaluated.
Google DeepMind announced the work on August 27 and described it as the first double-blind evaluation of a proprietary frontier-class model. MLCommons published a separate account the same day. The accompanying technical report identifies the model as Gemini 2.5 Flash Lite and says the pilot used private prompts from the AILuminate benchmark corpus inside an isolated NVIDIA H100 secure enclave on Google Cloud.
Why ordinary model testing has a custody problem
Private benchmarks are valuable because a model cannot be deliberately tuned to questions its developer has never seen. But testing through a normal provider-controlled interface sends those prompts into infrastructure operated by the model owner. Contracts and zero-logging promises can limit reuse, yet they do not create a technical barrier against accidental retention or later contamination.
Moving the model to the evaluator creates the opposite problem. Frontier model weights and inference code are expensive intellectual property and may carry security risks. A lab may therefore decline to hand them to an outside auditor, leaving the evaluation dependent on trust in the provider’s interface and data-handling practices.
The pilot placed both protected inputs inside a temporary confidential-computing environment. The technical report says the model weights and benchmark prompts were transmitted through encrypted channels after each party verified the hardware and software configuration. The evaluation ran in the enclave, bounded results were returned, and the environment was then decommissioned. MLCommons says AVERI used OpenMined’s secure computation system and a reserved set of AILuminate prompts that had not been exposed to Google DeepMind models.
What the cryptography changes
The first original contribution from TENS Magazine is to separate question secrecy from result validity. The enclave addresses custody: who can inspect the prompts, model weights, and execution environment. It does not by itself prove that the prompts represent real deployment conditions, that the scoring rule measures the intended risk, or that a result generalizes beyond one model version. A secret exam can still be a poor exam.
The second original contribution is a three-party accountability map. A credible double-blind test needs the model owner to commit a specific model, the benchmark steward to protect and maintain a meaningful test, and an independent evaluator to control execution and interpretation. Cryptographic attestation can verify what entered the protected environment, but it cannot replace institutional independence or benchmark governance outside that environment.
The third original contribution is a practical evidence ladder for buyers and regulators. A conventional benchmark score says how a system performed under disclosed conditions. A confidential benchmark adds resistance to rehearsal. A double-blind run adds separation between the model owner and test owner. The strongest claim is therefore procedural: the score was produced with fewer opportunities for either side to inspect or quietly shape the other side’s protected asset.
What the pilot does not settle
The technical report describes a hardware-rooted trust system, not a trust-free one. Participants still depend on the security of the processor, cloud infrastructure, firmware, container, and approved software stack. The report explicitly notes that vulnerabilities in the trusted computing base could undermine confidentiality or integrity. Reproducible builds and independent verification reduce that risk, but do not make it disappear.
The demonstration also should not be read as a sweeping safety verdict on Gemini. The public announcements focus on the evaluation mechanism, while the technical report describes selected private benchmark runs. No single test can establish reliability across every language, domain, adversarial strategy, model update, or real-world tool environment.
MLCommons makes a related point in its account: secrecy alone is insufficient. Benchmark stewardship still requires controlled prompts, documented scope, maintenance, and resistance to saturation over time. A protected execution box can preserve a test’s confidentiality, but the organizations around it must preserve its meaning.
A stronger chain of evidence
The most important change is not that cryptography produces a better score. It is that cryptography can make the path to that score more inspectable while revealing less. That could let independent organizations test closed models without forcing either side to surrender its most sensitive asset.
If the approach scales, future model reports could distinguish ordinary self-testing from externally governed, confidential, and double-blind evaluation. That would give readers a more precise way to judge evidence. The result would still need scrutiny, but the chain of custody behind it would be harder to dismiss as the model developer grading its own homework.
Sources: Google DeepMind; MLCommons; Double Blind Evals technical report by AVERI, Google, Singapore AI Safety Institute, OpenMined, and MLCommons.
Featured image: The Roadrunner supercomputer at Los Alamos National Laboratory. Photo: U.S. Department of Energy via Wikimedia Commons. Public domain. Center-cropped from 4,000 × 2,212 pixels to 3,932 × 2,212 pixels and resized to 2,400 × 1,350 pixels; no generative or substantive alteration. The photograph is illustrative and does not depict the evaluation described.
