Haru-4B Listens and Talks at the Same Time, Answering in About 65 Milliseconds on a MacBook

higurashi released Haru-4B on November 13, 2026, an Apache-2.0 full-duplex spoken dialogue base model built on Qwen3.5-4B and pretrained on more than 40 million hours of real speech, with code for PyTorch, MLX and vLLM-Omni.

Two friends laugh and talk over each other on a lilac park bench with a laptop between them, surrounded by chartreuse spring leaves.
Two friends laugh and talk over each other on a lilac park bench with a laptop between them, surrounded by chartreuse spring leaves.

higurashi has released Haru-4B, an open spoken dialogue model that listens and speaks simultaneously instead of waiting for a user to finish a turn. The company published the model on November 13, 2026 as a pretrained base model for research, with weights on Hugging Face under the Apache License 2.0 and code on GitHub under the same licence.

Most voice assistants still work in turns: the user speaks, the system detects silence, then it replies. Haru-4B is built for “full-duplex” conversation, where backchannels such as “mm-hm,” interruptions and overlapping speech are part of the exchange. higurashi says the model learns those behaviours from large volumes of real recordings rather than from synthetic dialogues or scripted turn-taking rules.

Three streams in one position

Haru-4B is built on the Qwen3.5-4B text backbone, with 32 layers and a hidden size of 2,560. It processes three synchronised streams: incoming user audio, the model’s own outgoing audio, and the model’s text. Rather than interleaving them token by token, Haru adds their embeddings together into a single position of the language model. According to the technical blog, this cuts context length to a third compared with interleaving while keeping the text stream unbroken.

Audio is handled as continuous values rather than discrete codebook tokens. Each frame is a 64-dimensional latent covering about 133 milliseconds of speech, which means the model runs at 7.5 frames per second. A small four-layer MLP head generates each audio latent from the language model’s hidden state in a single sampling step, trained with flow matching and then MeanFlow distillation. There is no separate acoustic transformer. The model card lists the VibeVoice acoustic tokenizer, causal and at 24 kHz, as its audio codec.

Text is aligned to audio in chunks, with padding and a dedicated end-of-padding token used to predict when speech should start.

Forty million hours of real speech

higurashi says Haru-4B was pretrained on more than 40 million hours of speech spanning over 160 languages, with nine primary languages: English, Spanish, Portuguese, German, French, Chinese, Japanese, Russian and Korean. The company deliberately excluded LLM-generated dialogues and TTS-synthesised speech from the mix.

Training ran in two stages. The first adapted Qwen3.5-4B to speech with speech recognition, speech and text continuation, and text-only rehearsal to limit forgetting. The second was full-duplex pretraining on more than 13 million hours of separated two-speaker dialogue, with samples up to 60 minutes long. Because most real conversations exist only as single-channel mixtures, higurashi built FastDialogueSidon, a joint speech separation and restoration model, to split them into aligned streams, and has released it publicly alongside Triton kernels for its Sidon restoration model.

Pretraining used 256 NVIDIA B200 GPUs for about eight days, on infrastructure built on Megatron-LM.

Latency on a laptop and a server

On a MacBook with an Apple M1 Max, higurashi’s native Swift app with custom Metal kernels reaches an end-to-end latency of about 65 milliseconds with int8 weights. The company’s breakdown puts one model step at 34.2 ms, with the rest spent on microphone input, handoff, an output margin and headphone output. That figure excludes audio framing, which the blog notes can add up to 133 ms of algorithmic delay at the 7.5 Hz rate. These are company measurements.

On the server side, Haru runs through vLLM-Omni, where audio encoding, decoding and one language model step take about 15 ms on an NVIDIA B300. The GitHub repository includes a PyTorch implementation with a web demo, an MLX implementation with a macOS app, and a vLLM-Omni plugin. The model card says a local GPU setup needs about 12 GB of memory and recommends wired headphones, since Bluetooth can add 100 ms or more.

Voices, characters and limits

A text “character card” sets persona and speaking style, and a reference clip of 3 seconds to 3 minutes sets the voice. Haru ships without voice clips, and the model card tells users to only use voices they have the right to use.

higurashi is explicit about the model’s limits. As a pretrained base model, it holds a natural conversation but is not a reliable source of facts and does not follow instructions like an assistant. The company says its small backbone limits response quality, and that it is already pretraining full-duplex models with more than 100 billion parameters, fine-tuning on native two-channel recordings, and working toward video input. A detailed technical report and training code are planned.

Development was supported by Amazon Web Services and by GENIAC, a generative AI programme led by Japan’s Ministry of Economy, Trade and Industry and NEDO.

Topics
  • #higurashi
  • #Haru-4B
  • #Full-Duplex
  • #Speech-to-Speech
  • #Open Weights
Raj M

Author

Raj M

Contributor

AI Systems Architect is a seasoned technology leader with over 15 years of experience in the IT industry working with Fortune 500 companies. With a solid foundation in multi-agent systems, open-source LLM infrastructure, and enterprise deployment, he excels at building scalable production-grade AI platforms.