Anthropic's FlashProve Proves Llama 4 Maverick Replies at 110 Tokens per Second Without Showing the Weights

Anthropic has open-sourced FlashProve, a research prototype that lets a provider commit to model weights and attach a short cryptographic proof that a reply came from those weights—and nothing else—without revealing the parameters. The Frontier Red Team post went up November 2, 2026. Code, Lean proofs, a technical report, and a protocol specification ship under Apache 2.0. The worked example is Meta’s Llama 4 Maverick, a 400B mixture-of-experts model with roughly 17B parameters active per token, using a third-party public 4-bit conversion (weights stay on Hugging Face under Meta’s license; Anthropic ships checksums and commitment artifacts).

The trust gap FlashProve targets is blunt: from text alone, a user cannot tell whether a router swapped in a cheaper model, an attacker rewrote the token stream, or an unauthorized heavier model is posing as a weaker one. FlashProve’s answer is a zkSNARK. Offline, the provider publishes a commitment—a short fingerprint that pins one exact weight set while keeping the weights hidden. That phase takes about 39 minutes on the measured setup and is good for on the order of 16 million requests before recommit. Online, for each request window, the prover emits a proof that the committed model produced those tokens. Completeness and soundness are the usual cryptographic pair; zero-knowledge means the proof leaks nothing about weights or which experts fired; succinctness means the client checks a small file on a CPU instead of re-running a frontier model.

Measured speed on one H100

Anthropic’s headline numbers use a single H100 with 24 CPU cores and about 290 GB of host RAM beside an 80 GB GPU. Each verifier request is a 1,024-token window; each proof checks one token. On a batch of 24 requests, the post reports about 223 seconds end-to-end, or roughly 110 tokens per second, with proof size about 18.78 MB. The client downloads the commitment once (~6.5 MB), then one proof per request. Verification takes about 17 seconds on 24 threads and 21 seconds on one thread. Against vLLM running FP8 Maverick on eight H100s, proving the same 24-request batch costs about 72× the per-GPU time of plain inference (about 94× at the highest vLLM load tested)—still orders of magnitude faster than prior open zkML systems Anthropic cites on the same 400B model, including VerInf’s roughly 0.02 tok/s and multi-hour verification on a 93.6 GB proof.

Three engineering choices drive that gap. Prover work scales with the experts that actually ran (~17B), not all 400B, while still hiding which experts were chosen. Requests are proved post-hoc in batches, opening the weight commitment once per batch. Layer segments stream from host memory so a single 80 GB GPU can host the prover. The construction follows an interactive-oracle-proof recipe compiled with hash functions; Anthropic notes post-quantum-oriented design in that family.

Lean-checked soundness, 1,024-token cap

Because cryptographic implementations fail quietly, Anthropic formalized top-level completeness, soundness, and zero-knowledge statements in Lean, with an interactive visualizer for auditors. A slow Lean verifier is proved to accept only what the soundness theorem accepts; a fast C++ verifier is for daily use and carries no formal guarantee beyond test agreement. The post is explicit about remaining gaps: the fast executable is not proved equivalent to Lean; proofs cover an integer, greedy-decoding model that can diverge from floating-point or sampled serving; architecture (as opposed to weights) is not hidden; context is capped at 1,024 tokens; side channels such as proving time are out of scope.

Anthropic frames applications from customer integrity and text provenance to tamper detection on inference servers and tooling that could support pacing or approved-model rules—without claiming FlashProve alone solves training-on-the-side or inline low-latency signing. Separately, the company says it already met a September 30, 2026 inference-verification roadmap goal with a different prototype that signs traffic and recomputes on a trusted server. FlashProve is the cryptographic research track released for others to harden. For API buyers and auditors, the concrete offer today is an open prover/verifier stack that makes “this reply came from these committed weights” a checkable file rather than a brand promise.