Microsoft's Mage-VL-Cowork Beats 27B Models at Desktop Control With Just 4B Parameters
Microsoft's 4B hybrid agent hits 66% on OSWorld-Verified by learning when to click a button and when to open a shell.
- Microsoft released Mage-VL-Cowork, a 4B hybrid computer-use agent under Apache-2.0.
- Scores 66.0% on OSWorld-Verified, best among 4B models, versus 72.4% human reference.
- Dynamically switches between GUI clicks (
computer_use) and shell commands (bash) per turn. - Finished a LibreOffice Calc task in 39 seconds versus 9:56 for Qwen3.6-27B, 15.3x faster.
- Trained via five-stage curriculum plus GRPO; RL alone added 10.16 points and cut tokens 25.4%.
- Full code, evaluation harness, and docker container on GitHub.
Microsoft has released Mage-VL-Cowork, a 4-billion-parameter vision-language model that operates desktop applications through screenshots, GUI actions and shell commands. On each turn, it selects either mouse-and-keyboard control or a persistent Bash and Python environment.
The model scored 66.0% on OSWorld-Verified with a 50-step limit, the highest result among the 4B-scale models in Microsoft’s comparison and 6.4 percentage points below the 72.4% human reference. The Apache-2.0 checkpoint occupies 9.5 GB in bfloat16 and can be self-hosted. That figure covers the weights alone; total memory use also depends on the serving framework, context length and runtime overhead.
One policy, two action channels
Mage-VL-Cowork exposes two tools within a shared turn format. computer_use provides 11 GUI primitives, including pointer, click and typing operations, with coordinates normalized to [0, 999]. bash provides a persistent shell and Python environment. The host harness, which connects model outputs to the operating system, executes the selected action.
GUI-only agents repeatedly process screenshots and reproduce structured edits through clicks. Shell access gives the model a direct route for tasks such as modifying spreadsheets, transforming files and validating outputs, while the GUI remains available for applications or controls without a practical command-line interface.
A LibreOffice Calc example in Microsoft’s report shows the potential latency difference for one task: merging ranges, adding labels and a column chart, then saving the workbook.
| Agent | Parameters | Steps | Time | Primary method |
|---|---|---|---|---|
| Mage-VL-Cowork | 4B | 6 | 39 seconds | Python workbook manipulation |
| Qwen3.6-27B | 27B | 39 | 9 minutes 56 seconds | GUI interaction |
The 15.3-times speedup applies to this Calc example and should be treated as task-specific rather than a benchmark-wide latency result.
A 4B model reaches 66%
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
