Clover-35B-A3B Writes Text in Blocks, Not Tokens, and DiffuSpace Put It Under Apache 2.0

Most large language models write one token at a time, left to right. Clover-35B-A3B-Preview drafts a whole block of text and then sharpens it in parallel passes. DiffuSpace released the model this week as the first in its Clover line, with weights on Hugging Face and ModelScope, code on GitHub, and an Apache 2.0 licence on the Hugging Face model card.

DiffuSpace’s launch post ties the team to the University of Hong Kong’s NLP group, which released Dream 7B in April 2025. The post says the Dream models have passed 2.7 million downloads on Hugging Face, and frames Clover as a test of a bigger question: whether a diffusion model can stay competitive with autoregressive models as it scales, while keeping its speed advantage.

Qwen’s backbone, reworked for denoising

Clover is not trained from a blank slate. The model card says it is a text-only model derived from Alibaba’s Qwen3.6-35B-A3B, and it keeps that model’s hybrid Gated DeltaNet and full-attention mixture-of-experts layout. That means 40 layers, 256 routed experts, top-8 routing and one shared expert per layer. About 3 billion of its 35 billion parameters are active per token. Context is 128K tokens (131,072) for prompt and response combined, and the model has a thinking mode.

The main engineering work is in how the Gated DeltaNet layers handle a block that is still being denoised. According to the launch post, text that has already been generated is compressed into the network’s recurrent state, while tokens inside the current block attend to each other through bidirectional softmax attention. The memory readout joins those block tokens in the same softmax, so the model learns when to lean on history and when to use the block itself. DiffuSpace also adds learnable block-size calibration and a zero-initialised convolution branch for right-hand context.

In the Hugging Face implementation, each 32-token block starts as masked tokens. Positions whose confidence clears a threshold are filled in parallel, and if none qualify, the single most confident position is filled so generation keeps moving. A finished block then becomes part of the prefix for the next one.

The numbers, as DiffuSpace reports them

All benchmarks and speeds are DiffuSpace’s own and have not been independently reproduced. The team reports:

• Agentic coding and tools: 73.6 on SWE-bench Verified, 51.7 on Terminal-Bench 2.1 and 65.3 on τ³ Overall. DiffuSpace says Clover was top among the models it compared on all six agent evaluations, including the diffusion models LLaDA2.2 flash and Mercury 2.5 and the autoregressive Qwen3.6-35B-A3B and Qwen3.5-35B-A3B.
• General ability: 83.8 on GPQA Diamond, 95.8 on ZebraLogic, 72.0 on AA-LCR and 84.8 on MMLU-Pro.
• Speed: 680 tokens per second on one NVIDIA GB300 at BF16 and batch size 1, measured on internal agent workloads. On one H200 it reports 515 tokens per second against 365 for a Qwen setup accelerated with DFlash2.
• Consumer hardware: an INT4 build running on a 16 GB RTX 5070 Ti at 101.8 tokens per second on MATH-500, against 48.5 for similarly sized MiniCPM and Qwen models in DiffuSpace’s tests.

Running it today

The Hugging Face path needs Transformers 5.19.0, PyTorch 2.10 or later and trust_remote_code=True. The BF16 weights take about 69 GB of GPU memory before inference state. The model card ships default repetition controls, plus gradual biases that nudge long answers and long thinking phases to close. DiffuSpace says very long generations “can sometimes become repetitive.” The team recommends its own SGLang implementation for serving and evaluation, and also supports llama.cpp, including self-speculative decoding.

Optional block-level expert selection limits routing to a shared subset of experts covering 90% of the aggregated routing mass per block by default.

The Hugging Face repository was created on October 24 and last updated on October 28. The GitHub repository was created on October 27 and had 85 stars early on October 29, while the model page showed 99 likes.

Licence and what is still missing

The model card lists Apache-2.0, and its attribution notes say the upstream Qwen assets are copyright Alibaba Cloud under the same licence, with file-level modification notices in ATTRIBUTIONS.md. DiffuSpace’s post says both weights and code are released under an Apache licence, an OSI-approved open-source licence, though the model card does not describe the training data.

The “Preview” label is meaningful: DiffuSpace says the agent harness and desktop coding client it is building around Clover are still in development.