Kandinsky 6.0 Video Puts Lip-Synced Speech and Sound Into a 29B MIT-Licensed Generator

Kandinsky Lab has released Kandinsky 6.0 Video, a pair of open diffusion models that generate short video clips together with a matching soundtrack—dialogue, music and ambient sound—with characters’ lip movements aligned to their speech. The lineup has two sizes: Kandinsky 6.0 Video Pro at 29 billion parameters and Kandinsky 6.0 Video Lite at 3 billion. Code, checkpoints and a Diffusers integration are published under the MIT license, which permits commercial use without a separate agreement.

Most open video generators still produce silent footage, leaving creators to add voice, effects and music in a second tool and then fight to line them up. Kandinsky 6.0 Video treats picture and sound as one generation problem. A user can describe a scene in text, including what should be heard, or upload a first frame and describe the audio track that should accompany what happens next.

Two streams that listen to each other

The technical report, posted to Hugging Face Papers under the identifier 2610.05608, describes a dual-stream CrossDiT architecture. A video stream inherited from Kandinsky 5.0 is connected to a newly trained audio stream through bidirectional cross-attention, so each modality can condition the other for timing and meaning. That design is what lets a door slam land on the frame where the door closes, or a syllable match the shape of a mouth.

Training ran in stages. The team first trained the audio stream from scratch on large audio corpora, then trained both streams jointly on paired audio-video data while trying to preserve the quality each stream achieved alone. Supervised fine-tuning, reinforcement-learning post-training and distillation followed. The authors say reinforcement learning cut the Pro model’s speech word error rate by 47%, a measure of how intelligible generated dialogue is.

Clip length, resolution and sample rate

Both models output five-second clips with 44 kHz audio in two modes: text-to-audio-video and image-to-audio-video, where a reference image conditions the opening frame. The default Pro example on the model card renders 121 frames at 864 by 480 and encodes them at 24 frames per second. A separate super-resolution pipeline upscales frames by 2, 2.25 or 4 times using tiled diffusion, reaching Full HD at 1920 by 1080. The audio is carried over from the generation step, since the upscaler works on frames only. The developers say they plan to extend clip length beyond five seconds over time.

The released family covers more than two files. The Pro and Lite lines each ship a main checkpoint, a pretrained checkpoint and a distilled variant that runs in 10 steps. Two upscalers are available: a flow-matching version that runs four steps per tile and a distilled version that runs two. The Pro pipeline pairs a Qwen2.5-VL text encoder with a CLIP encoder, uses a HunyuanVideo-style video VAE, and decodes sound through an MMAudio VAE and a BigVGAN-style vocoder. An optional prompt-expansion setting lets the Qwen2.5-VL encoder rewrite a terse prompt into a detailed one, at the cost of extra latency but no extra weights.

Where it stands against other generators

Kandinsky Lab reports that, in side-by-side human evaluation, the Pro model clearly beats its predecessor, Kandinsky 5.0 Video Pro. Against LTX 2.5, evaluators preferred the new model for visual quality, motion realism, visual prompt following, artifact reduction and overall task completion, and the team reports a statistically significant edge in speech quality. On the VABench suite, the Pro model posted the best results among LTX 2.5 and the two Kandinsky 6.0 sizes on speech quality, audio aesthetics, audio-video alignment, lip-sync, desynchronization and visual realism. The authors describe it as competitive with leading audio-video systems rather than ahead of all of them, and the comparisons are their own.

Getting it running

The models plug into Diffusers through a Kandinsky6TI2VAPipeline class, and the developers list support in ComfyUI, FastVideo, SGLang and vLLM-omni, so teams can try the models in existing tooling without writing integrations. Generation can be switched to video-only, skipping the audio branch. The model card examples use bfloat16 weights with CPU offload enabled, a sign that the 29B Pro checkpoint is aimed at well-equipped workstations or cloud GPUs, while the 3B Lite model targets lighter hardware.

For voice and media developers, the release is notable less for any single benchmark than for licensing. Synchronized speech and sound have mostly lived behind closed APIs. A permissively licensed model that produces both, along with its training report and distilled fast variants, gives studios and app builders a base they can fine-tune, self-host and ship inside products.