Nous Reports State Framing Fell From 22% to 0.9% on Its Arete-27B Evaluation

Nous Research has released Arete-27B, a fine-tune of Alibaba’s Qwen3.8-27B built to remove the refusals, deflections, and Chinese Communist Party framing that the lab says it found in the base model on politically sensitive topics. The weights went up on Hugging Face on November 6, 2026 under Apache 2.0, the same license as the base model. Alongside them, Nous published the 803-prompt evaluation it used to measure the change and said it is publishing the training data. The lab frames the release as a test of a broader claim: the values built into an open-weight model sit near the surface and can be changed with a modest amount of supervision.

Nous says it picked Qwen3.8-27B because it was the strongest model it could run locally, and because it showed many of the behaviors the lab wanted to change.

How the evaluation works

Before training, Nous built a benchmark covering seven value categories: freedom of expression, state ideology, state versus free market, education, data privacy and surveillance, democracy, and the environment. Each prompt comes from English-language statutes, news, or research reports, with no single source contributing more than 13 prompts. Each one carries two to four required facts from a dated source, plus the official phrasing the Chinese state uses on that topic.

A judge model, Grok 4.7, scores each answer without knowing which model wrote it. Fact recall runs from 0 to 5, and a single fact the answer contradicts sets the score to zero. A separate framing score marks whether the answer adopts the state’s account as its own. Simply quoting the official line to identify it does not count, and any framing flag must cite the exact phrase. Nous tuned the rubric on a hand-graded 25-prompt pilot.

Small training run, large shift

The recipe is deliberately plain. Nous generated about twenty thousand prompts with a deduplication pipeline, removed any that overlapped the evaluation set, and collected answers from a teacher model. It chose Inkling, a Thinking Machines model, as the teacher. The lab says it considered NVIDIA’s Nemotron 3 Ultra but found evidence of similar problems on China-related topics. Training was supervised fine-tuning only, with no reinforcement learning, preference optimization, or architecture changes. It covered about 52.8 million tokens at 64K context over two epochs and 1,202 steps. All language-model weights were trained, and the vision encoder was frozen because the data is text-only.

On Nous’s evaluation, the share of answers with state framing fell from 22% to 0.9%, and the mean fact score rose from 2.32 to 2.98 out of 5. Answers scoring zero dropped from 359 to 216 of 803. The biggest change was in freedom of expression, where the fact score climbed from 0.89 to 2.44 and the framing rate fell from 60% to 1%. For comparison, the same table lists GLM 5.3 at 15% and DeepSeek V4.1 Flash at 16%, with Western models such as Mistral Large 4 and Gemma 4 31B between 0.3% and 0.5%.

Nous says capability held. On its internal Hermes Bench, Arete scores 65.60% against 65.77% for the base model, which the lab calls noise. On Terminal-Bench 2.1 it posts 70% against 67%, though Nous says it does not take credit for that gain because its inference setup differs from official runs.

One judge and mixed decoding settings

The comparison uses mixed decoding settings. Arete and the base Qwen model ran at temperature 0 with thinking off, while most other rows used 0.6. At 0.6, Arete scores 2.88 and the base model 2.24. One thinking-on result on the blog is marked provisional, based on two of five runs. A single LLM judge does all the scoring, and Nous both designed the benchmark and trained the model. The lab also says it has not removed every behavior tied to the base model’s viewpoint, only what it calls a likely root cause.

The broader point Nous makes is that the values in any open model are one fine-tune away from changing, in either direction. It asks other labs and researchers to run the open evaluation on their own models and report what they find.