Nokia’s AnyJev Turns Open LLMs Into Training-Free Typed Decision Engines

Nokia Applied Research released AnyJev as an Apache 2.0 Python library that turns an open-weight language model into a typed decision engine: given a state and a set of structured questions, it returns an answer and a probability from one prefill of the model’s next-token distribution—no generation loop, no output parsing, and no fine-tune required for the default path. PyPI shows anyjev 0.0.1 and 0.0.2 uploaded on September 21, 2026; the GitHub trees under nokia-applied-research/AnyJev and MorrisZJ/AnyJev carry Apache License 2.0 files and README authorship from Nokia researchers in Sunnyvale with a Tencent Hunyuan collaborator. The project states plainly that it is not affiliated with TypeSafe AI’s commercial Jev product even while it benchmarks against that interface.

The practical problem AnyJev targets is familiar to anyone who has tried to gate an agent with “just read the logits.” Ask a model which of twenty billing intents applies, or whether a proposed tool call is irreversible, and a raw max_tokens=1 readout will often flip when you reverse the option list or lean on the model’s prior preference for “Yes.” AnyJev’s default L0 path applies cyclic-shift permutation marginalization to cut position bias and a label-free prior correction to cut content-free label bias. L1 adds temperature scaling when you have roughly 100–500 labeled examples per question. An unreleased 0.1.0 line documented in CHANGELOG entries dated around September 22 adds L2: closed-form per-question heads fit on hidden states at about two-thirds model depth, with routing across paraphrases and optional unlabeled adaptation. Every Decision object carries a mandatory level field (raw, L0, L1, or L2) so downstream code can refuse to act on the wrong contract.

The API is intentionally small. Question.choice, Question.noul, and Question.score declare the typed asks; a Decider wraps an HFBackend (transformers) or VLLMBackend (OpenAI-compatible server). One call can answer a routing choice, a destructive-tool Boolean, and a completion score against the same conversation state. Benchmarks shipped in the repository regenerate Markdown and JSON from measured runs rather than hand-typed tables. On Qwen3-8B with a 20-way BANKING77-style task, the README’s summary row shows option-order flip falling from about 0.23 under raw logits to about 0.07 under L0, accuracy rising into the low 0.80s, and the share of traffic that can be auto-decided at a 5% error budget jumping from single-digit percentages under raw confidence to roughly half after calibration—exactly the lever agent runtimes care about when most “0.9” scores are not trustworthy enough to skip a human.

AnyJev also publishes harder comparisons. On LocalLLaMA/typed-decisions it measures zero-shot and temperature-scaled open models against Laya checkpoints and published Jev numbers, separating argmax accuracy from ECE, Brier, and soft accuracy so a fine-tuned specialist that wins accuracy but loses calibration does not hide. A maze harness adapted from NanoJev is included as a negative-result stress test: even strong readouts struggle to beat majority baselines on local edge questions, which the authors report rather than bury. Optional extras install torch/transformers for the HF path and datasets/scikit-learn for benches; the wheel itself stays lean (numpy only) so production agents can bring their own serving stack.

Installation stays deliberately boring. pip install "anyjev[hf]" pulls the library plus a transformers backend; pip install anyjev alone leaves serving to the caller. The benchmark suite is not stuffed into the wheel—it expects a checkout so datasets, committed result JSON, and provider shims stay reproducible. CHANGELOG notes around the September 22 reproduction pass freeze L1 priors inside artifacts, record batch size and git commit in result files, and fix content-free probes so they no longer perturb batch boundaries. Those details are the sort of engineering that separates a demo notebook from something an agent runtime can pin. Roadmap items already named in-tree include span readout for large option sets, conformal abstention, and broader model rows beyond the Qwen family used in the first tables.

For Writeble’s ai-agents desk, AnyJev is the rare September drop that is both open and operationally sharp. Agent frameworks already need reliable System One decisions—route this ticket, allow or block this tool, score task completeness—without spinning up a specialist model per question. A training-free Apache library that makes open LLMs emit leveled, calibrated distributions, with reproducible benches and an explicit non-affiliation notice, is more useful than another agent demo that prints free-form prose and hopes a regex holds. Teams can pip install "anyjev[hf]", point it at a Qwen checkpoint they already host, and decide whether L0 alone is enough before investing in L1 labels or the emerging L2 heads.