Microsoft's 4B Mage-VL-Cowork Decides Each Turn Whether to Click the Screen or Open a Shell, and Scores 66% on OSWorld-Verified

Microsoft has published the weights of a 4-billion-parameter computer-use agent that can either work through the screen or drop into a terminal, choosing per step. The model, microsoft/Mage-VL-Cowork, went up on Hugging Face on October 30, 2026, under the Apache-2.0 license, with code in the microsoft/Mage GitHub repository and a technical report. The model card sums up the pitch in one line: “One compact 4B policy that knows when to click — and when to code.”

The card lists the developer as the Microsoft Mage Team. The backbone pairs Mage-ViT, a vision encoder the team trained from scratch, with Qwen3-4B-Instruct-2507. Weights are about 9.48 GB in bfloat16.

Two channels in one policy

Most computer-use agents work only through screenshots and mouse and keyboard events. The Mage team argues that this “pay[s] a heavy tax,” because each screenshot costs thousands of tokens and structured edits get rebuilt from dozens of clicks. Mage-VL-Cowork exposes two tools: computer_use, with 11 GUI actions on a normalized 0–999 coordinate grid, and bash, a persistent shell and Python environment whose output comes back as text rather than as an image.

The model picks the channel itself, “with no external orchestrator.” The card says it uses the screen for visual layouts, canvas editing and live application state, and the shell for programmatic edits to spreadsheets, documents and slides and for checking results. In one example, it merges cells in an .xlsx file by writing a short openpyxl script instead of opening LibreOffice Calc.

To keep long tasks within a 32,768-token context, the model keeps the three most recent screenshots and four full responses, and folds older turns into one-line Action: summaries.

Pruning the teacher before distilling

The training story starts before the student model. The team “evolved” the harness its teacher model used, removing overlapping tools such as xdotool and verbose accessibility-tree dumps and pre-installing libraries like python-pptx, openpyxl and pandas. With GPT-5.4 as the fixed teacher on 672 held-out multi-application tasks, the pruned harness scored 92.0% against 89.6% for a version with accessibility data, in 29.2% fewer steps. A 4B student trained on each teacher’s runs scored 37.6% versus 25.1%.

Task data came from Mage-CUA-Forge, a generator that builds reusable workflows per skill rather than running an open-ended agent loop per task. The card reports 293,806 tasks across 6,915 atomic skills and 318,837 verified hybrid trajectories, with recurring cost per task falling from $0.906 to $0.158.

Training ran in five stages: GUI-oriented midtraining, GUI agent fine-tuning, coding and terminal fine-tuning, hybrid fine-tuning, and online reinforcement learning with GRPO. The card lists 7.79 ZFLOPs of nominal compute on 32 NVIDIA B200 GPUs plus 1,120 CPU sandbox slots for the RL stage.

Where 66% sits on the leaderboard

The headline number is 66.0% on OSWorld-Verified, the average of two full runs (65.33% and 66.75%) over 361 tasks that exclude Google Drive, with a 50-step limit. The team calls it “the best performance among 4B models.” In its own comparison table, the 4B Fara1.5-4B scores 13.9% on the same desktop benchmark, while the 9B ScaleCUA-Qwen3.5-9B scores higher at 68.7%, and GPT-5.4-mini reaches 69.6%. Frontier figures quoted in the table, such as 78.7% for GPT-5.5, come from a column without step counts.

The RL stage added 10.16 points over hybrid fine-tuning while cutting model calls per task by 13.0% and output tokens by 25.4%. On a browser test the team built, WebVoyager-Static, the fine-tuned version scored 58.6%; Fara1.5-4B, a browser specialist, scored 68.0% there.

All of these results are the team’s own. No outside group has reproduced them yet.

Running it, carefully

The model uses a custom llava_onevision2 architecture, so loading requires trust_remote_code=True, and the card warns against assuming drop-in compatibility with standard Qwen3-VL classes. The team supplies a pinned SGLang Docker image and an osworld_eval/ package with its hybrid agent code.

Because the agent can run arbitrary shell commands, the card’s safety section calls for “Mandatory Sandbox Isolation” in a container or VM, human confirmation before irreversible actions, no personal SSH keys or tokens in the workspace, and network allow-listing to limit prompt injection from web pages.

For developers who want computer use on modest hardware, the release gives a permissively licensed starting point and a detailed recipe. Whether the hybrid approach holds up outside the team’s benchmarks is the open question.