Iris-3B Drops the VAE Entirely, and Its Authors Admit the Payoff Has Not Arrived Yet

Sperid Labs has released Iris-3B, a 3-billion-parameter diffusion transformer that generates images directly in pixel space, with no variational autoencoder and no latent space in between. The weights, the full training and fine-tuning code, an inference script and a Hugging Face demo are public under the Apache 2.0 license. The accompanying paper, posted to arXiv on October 7, 2026 as 2610.09450, is unusually candid: its central hypothesis did not hold up.

Why skip the latent space at all

Most modern image generators, from Stable Diffusion to FLUX, do not draw pixels. They work in a compressed latent space produced by a VAE, then decode the result. The compression makes training and sampling cheaper, but the paper calls the VAE “lossy” and “texture-biased”: fine detail that does not survive the encoder can never be recovered by the generator.

The Iris team wanted to test a specific claim that follows from that. If a model learns to produce every pixel itself, its internal representations should be richer, and therefore more useful when the same network is repurposed for dense vision tasks where detail matters, such as depth estimation or super-resolution. The project README frames this as exploring pixel-space generative models as an alternative to vision foundation models such as DINOv2.

How the model is built

Iris-3B counts 3 billion parameters, excluding a frozen text encoder. Its trunk has 24 blocks at width 2,560: eight dual-stream blocks with separate image and text weights sharing joint attention, followed by 16 single-stream blocks. It uses a patch size of 16. Instead of a VAE decoder, each patch is turned back into pixels by four pixel-transformer (PiT) blocks, a head design borrowed from PixelDiT, which apply per-pixel modulation while attending across patches.

Text conditioning comes from Qwen3-VL-4B-Instruct, kept frozen, with 12 of its hidden layers combined by a learned layer-attention adapter. Training uses rectified flow with v-prediction.

The model was pretrained from scratch through a 256, then 512, then 1,024-pixel curriculum. The released configs list 365,000 steps at 256 pixels with representation alignment against DINOv2, then stages at 512 and 1,024 pixels with multi-aspect data, and a final supervised fine-tuning stage ending at 665,000 steps. Optimization uses a hybrid of Muon for hidden weight matrices and AdamW for embeddings, norms and biases.

The result the team did not expect

To test the downstream claim, the researchers fine-tuned Iris-3B for monocular depth and for 4x image restoration. They also took a second route, converting a pretrained latent model, FLUX.2 Klein base 4B, into pixel space, and fine-tuned that too.

The abstract states the outcome directly: “We find no significant improvement from using a pixel-space generative prior.” With a matched depth recipe, Iris-3B was level with the latent FLUX.2 Klein, and the converted pixel-space FLUX.2 Klein fell behind it. On 4x DIV2K restoration, neither pixel model beat a latent FLUX.2 Klein fine-tune. The paper says it documents the recipes, the failure modes and the remaining confounds behind what it calls a negative result.

Negative results rarely ship with full weights and code. Here, other researchers can rerun the experiments, swap recipes and check whether the confounds the authors list explain the gap.

Where Iris-3B does deliver

The generation side is stronger. According to the paper, Iris-3B shows that pixel-space pretraining with the PiT head scales to 3 billion parameters and to text-to-image quality competitive with latent models, matching Qwen-Image on the OneIG benchmark under the official evaluators at 1,024 by 1,024 pixels.

Both fine-tunes ship in the same Hugging Face repository, in depth/ and upscaler/ folders. The depth model returns relative, affine-invariant log depth rather than metric depth. The restorer is a one-step model that tiles large images in overlapping 1,024-pixel blocks and applies a wavelet color fix.

What it takes to run

The main checkpoint is about 12 GB, and each fine-tune is roughly the same size. Default sampling runs 100 DPM-Solver++ steps at 1,024 by 1,024 with a guidance scale of 3. The demo notes that all three models together need about 45 GB of GPU memory; with CPU offloading enabled, peak usage drops to around 20 GB. The code requires Python 3.11 or newer and PyTorch 2.7.1 or newer.

Users should expect research-grade behavior, and 100-step sampling is slow next to the distilled few-step generators now common in production. For anyone studying whether VAEs are a bottleneck, though, Iris-3B offers something scarce: a fully open, large pixel-space baseline with an honest account of where it falls short.