SenseNova-Loop-9B Runs 24 of Its Layers Twice to Act Like a Bigger Model

Most language models read each layer once and move on. SenseTime’s new open model sends a big block of its layers around a second time, betting that extra depth at run time can stand in for extra parameters. The SenseNova team released SenseNova-Loop-9B-Preview on October 30, 2026, with weights on Hugging Face, code on GitHub and an Apache 2.0 licence.

The model takes text and images and produces text. SenseNova describes it as built for long-context and agentic work: reading documents, writing code and calling external tools over several steps. The model card’s showcase, generated with SenseTime’s Raccoon Work product, leans on slide decks, motion design and data-analysis visuals.

Fifty-six layer passes from thirty-two layers

The core idea sits in the model’s layer stack. SenseNova-Loop has 32 distinct decoder layers. Input goes through layers 0 to 3 once, then the block of layers 4 to 27 runs twice in a row with the same weights, the second pass taking the first pass’s output as its input. Layers 28 to 31 and the output head finish the job. That adds up to 56 decoder-layer computations per forward pass while only 32 sets of layer weights are stored.

The model card lists about 9 billion total parameters and about 14.6 billion “inference parameters,” the effective size once the repeated block is counted twice. Other published settings include a hidden size of 4,096, a feed-forward size of 12,288, a vocabulary of 248,320 tokens and 16 attention heads with 4 key-value heads.

Two other pieces sit around the loop. The attention stack is hybrid: every group of four layers has three linear-attention layers and one full-attention layer, and the loop boundaries line up with those groups so the pattern holds on both passes. Inside the looped section, the team uses what it calls two-stream manifold-constrained hyper-connections, or mHC. Two residual streams carry information around the repeated layers, and learned mappings decide how each layer reads from them and writes back. The stream-mixing map is constrained with Sinkhorn normalization, run for 20 iterations in the published configuration.

Training recipes mix text, images, code and tool trajectories at sequence lengths up to 262,144 tokens.

How it compares, by SenseNova’s chart

The benchmark chart on the model card compares the 9B model with Qwen3.5 and Qwen3.6 models, Nanbeige 4.2-3B and IQuest-Coder-V1-40B-Loop-Thinking. Several results favour SenseNova-Loop:

• BIRD-Dev (text-to-SQL): 71.38, against 69.84 for Qwen3.5-27B.
• WorkBuddy Bench: 66.89, against 61.98 for Qwen3.6-35B-A3B and 60.53 for Qwen3.5-27B.
• AutomationBench: 13.72, against 10.17 for Qwen3.5-27B.
• RaccoonPPTBench: 70.70, against 64.30 for Qwen3.6-35B-A3B. A footnote says this is an internally developed benchmark for presentation generation.

The same chart shows where it does not lead. On Terminal Bench 2.1 it scores 40.45, level with Qwen3.5-35B-A3B and Qwen3.6-35B-A3B but behind Qwen3.5-27B at 47.19 and Nanbeige 4.2-3B at 49.44. On τ³-Banking it posts 18.56, just under Qwen3.5-27B’s 19.59. All of these are SenseNova’s own figures, and the technical report is still listed as “TBD.”

Running and tuning it

The documented setup uses Transformers 5.18.0 and PyTorch 2.11.0 with CUDA 13.0, and the model loads with trust_remote_code=True because the looped architecture needs custom code. The card notes that any compatible inference engine must load the loop range, pass count and mHC routing parameters and reproduce the same execution order.

For serving, the card gives a four-GPU tensor-parallel example for SenseTime’s LightLLM framework and points to an open SGLang pull request (#47281) that adds the architecture. Multi-token prediction is available now for faster decoding; two other speed-up methods, DSpark and DFlash2 draft models, are still training, although recipes for training them are already published in the team’s SenseSpec repository. Fine-tuning runs through a fork of torchtitan, with an example script for a single node of eight GPUs that also covers DPO and MTP training.

Why the loop matters

Small open models usually gain ability by adding parameters or training longer. SenseNova-Loop tries a third lever: spending more compute per token on the same weights. That keeps the download and memory footprint of a 9B model while asking the GPU to do more work on each forward pass, a trade that suits teams constrained by VRAM more than by latency.

Whether the gains hold up outside SenseNova’s chosen benchmarks, several of them agentic and one internal, is the open question for this preview. The weights and training scripts are public, so others can now check.