ShadowPEFT Lands in Hugging Face PEFT as a Stateful Alternative to LoRA-Style Adapters

ShadowPEFT is now wired into Hugging Face’s PEFT library as a first-class tuner, according to a September 15, 2026 Hugging Face Blog post by Zongxi Li, Xianming Li, and collaborators. Unlike LoRA-style low-rank deltas on selected linears, ShadowPEFT runs a compact shadow network with a persistent hidden state that injects into and updates from each frozen Transformer block.

Integration follows PEFT’s usual pattern: ShadowConfig, a BaseTuner model, and get_peft_model. Checkpoints use standard save_pretrained / from_pretrained. The authors say the method is merged on PEFT main and will ship in the next release; until then, install with pip install --upgrade git+https://github.com/huggingface/peft.git. After training, unload_shadow() can detach a standalone shadow model for edge-style inference.

Author-reported comparisons

On MetaMathQA→GSM8K with Llama-3.2-3B, the post’s table lists ShadowPEFT at 48.1% test accuracy versus LoRA 46.9% and DoRA 46.2% at similar trainable-parameter counts, with a smaller checkpoint but higher peak memory. On a DreamBooth run with FLUX.2-klein-base-4B, the same post reports higher DINOv2 subject similarity and lower drift than LoRA/DoRA. Treat those figures as author-reported until independent replications appear. The method paper is cited as arXiv:2604.19254.

Primary sources are the September 15 Hugging Face Blog post, the huggingface/peft repository, and the linked PEFT ShadowPEFT documentation page.