H Company Open-Sources NeoMME Multimodal Encoders for Visual Document Retrieval

H Company published NeoMME, a family of 260M and 800M multilingual multimodal encoders, in a Hugging Face blog post dated September 3, 2026. Unlike generative vision-language models that bolt a pretrained vision tower onto a causal decoder, NeoMME uses one bidirectional Transformer for text tokens and raw 32×32 image patches, trained from scratch with a masked discrete-diffusion objective. All checkpoints ship under Apache 2.0 with day-zero support in Hugging Face Transformers; the technical report is arXiv:2609.01657.

The authors argue that visual document retrieval does not need the parameter and compute overhead of a full generative VLM. NeoMME therefore drops the separate vision tower and causal language model, keeps a 16,384-token bidirectional context (enough for up to two standard 4K UHD images), and mixes multilingual text, code, mathematics, natural images, and document pages in pretraining.

NeoMME-Retriever on the ViDoRe Pareto frontier

Fine-tuned NeoMME-Retriever adds jointly trained dense and late-interaction heads and returns both embeddings in a single forward pass, following ColPali’s page-image retrieval approach. On ViDoRe v3, NeoMME-Retriever-260M reaches 0.523 nDCG@10—the highest score among evaluated models strictly below 800M parameters—while the 800M variant reaches 0.556. Both sizes sit on the model-size Pareto frontier reported in the blog.

At a matched 2048×2048 image input on one NVIDIA L40S, the 260M retriever encodes about 51 pages per second, roughly twice ColModernVBERT’s 26 pages per second in the same comparison. Hierarchical token pooling plus asymmetric quantization compresses late-interaction index storage from about 1.5 MB to 6 kB per page (255× smaller) while retaining more than 95% of baseline nDCG@10 on ViDoRe v3.

Open checkpoints and visual RAG path

Hugging Face hosts backbone and retrieval checkpoints in the Hcompany NeoMME collection, with separate Sentence Transformers dense and late-interaction fine-tuning variants. The blog also points to a visual RAG demo Space that retrieves page images and feeds them to a vision-language model without OCR-first chunking.

Primary sources for this brief are H Company’s September 3 Hugging Face blog post on NeoMME, arXiv:2609.01657, and the public NeoMME model collection on Hugging Face.