Mirendil's First Model, Fern, Bets on a 10-Million-Token Window Instead of More Parameters

Most labs that want a smarter model add parameters. Mirendil, a young AI lab, is asking how much a model can hold in view at once. On October 26, 2026, it introduced Fern, its first model. Fern was pretrained from scratch, and its architecture, training and serving support “up to 10M tokens” of context, which the company equates to about 23 MB of code. That is roughly half a million lines of code, “a codebase on the scale of the Megatron-LM library,” or, in Mirendil’s words, “days of an agent’s work, all held in context at the same time.”

The first public version is Fern 0.1. Mirendil describes it as its base model “after a short round of post-training” and says “new versions ship frequently.” For now it can’t be used: the launch thread on X ends with a link to “request early access,” and no pricing, API documentation or weights have been published.

Context as a second scaling axis

Mirendil’s launch post makes a simple argument. A model’s capabilities grow with compute, and there are two ways to spend it. More parameters give a model “more static knowledge” and deeper reasoning steps. More context lets it use more task-specific information, such as its own past work, long specifications, many documents and large amounts of code, all “look[ed] at directly” rather than summarized or retrieved in pieces.

To back this up, the company shows that next-token prediction keeps improving as context grows, “all the way to 10M tokens,” and reports results on RULER, a standard long-context benchmark. Mirendil presents RULER as only “a basic test of effective access to the long context” and says “more comprehensive long-context results are coming soon.” Its launch thread also includes a chart of Fern 0.1 on agentic benchmarks. The company frames that chart as an early snapshot after limited post-training, and the scores are its own.

Built by one researcher and a lot of agents

Fern’s backstory arrived five days earlier. On October 21, Mirendil published a post titled “Pretraining at scale without a pretraining team” about Mirendil-Base, the model underneath Fern. The company says one researcher built it in 3.5 months for less than $10 million in compute. That figure is “all-in,” covering research, small-scale experiments, the final training run, the agents’ token spend and data processing. Mirendil says AI agents “wrote over 99% of the code, ran experiments and diagnosed failures,” while the researcher “decided what to try and what to trust.”

According to the post, the code was written from scratch on standard libraries like PyTorch and JAX. The training data was only “organic, publicly available data” from GitHub, web pages and public datasets, with no purchased or licensed data, no distillation and no synthetic data.

Mirendil backs the quality claim with a bits-per-byte comparison, a tokenizer-independent measure of how well a model predicts text where lower is better. On seven external repositories published after the models were trained, Mirendil-Base averages 0.222. That beats DeepSeek-V4-Pro-Base (0.243), Nemotron-550B-Base (0.247) and GLM-5.3 (0.242), but trails Kimi-K3 (0.193). Mirendil notes that Kimi-K3 and GLM-5.3 are post-trained models, which usually score slightly worse on this measure than their base versions.

Two bigger models in the pipeline

Fern is the smallest of three. Mirendil says Mirendil-Base sits in the 100-billion to 500-billion-parameter range. A Large model of 0.5 to 2 trillion parameters was about 80% through training as of the October 21 post, and an XL model of 2 to 10 trillion parameters was about 19% through. The Fern launch page already shows two more capable tiers built on those larger bases, with their context, input types and access left blank.

Baseten, which says it will serve Mirendil’s models, posted on October 23 that it ran Mirendil-Base-Large on its own private codebases. According to Baseten, the model “had the lowest bits per byte” of any model it tested. Baseten didn’t publish the scores.

What it’s for, and what’s missing

Mirendil is targeting the work AI labs do on themselves, not general chat. It lists auditing pretraining shards for duplicates and benchmark contamination, debugging a training run from its logs, reading agent and training traces end to end, and agentic coding across large repositories.

A number of key details are still missing. There are no published latency or memory figures for a 10-million-token prompt, no price, and no indication of whether weights will be released. Mirendil also hasn’t shown how well Fern reasons over that much text, as opposed to simply retrieving from it. The October 21 post promised “independent evaluations, post-training benchmarks, and API access” in the future. Until those arrive, Fern’s 10-million-token window is a claim the company hasn’t yet let outsiders test.