Mage-VL-Cowork: A 4B Agent That Chooses Clicks or Code

Open Source Factory · 2 November 2026 · Release analysis

English 3D motion graphic: Open Source Factory. Newly composed original music and sound design: “Click. Code. Verify.”, 128 BPM. Concept animation, not a model run. Data sources: Microsoft Mage Team; checked 2 November 2026.

A desktop agent can spend twenty clicks fixing a spreadsheet, or edit the file with a short script. Mage-VL-Cowork is interesting because it learns to choose the route. Microsoft’s 4B-class model combines screenshot-based interaction with terminal execution. The useful question is whether that choice produces a file you can actually reopen and use.

My answer from the release is cautiously positive: the design makes sense, and the team supplies concrete desktop demonstrations. But the headline 66% is a reported benchmark result under a specific setup. It is not a promise that two out of three tasks on your PC will work.

Scope, checked 2 November 2026: this article reviews Microsoft’s release, reference agent, configuration and file metadata. I did not run model inference or reproduce its desktop benchmark. The 3D film is an editorial explanation. Reported scores and demonstrations below belong to Microsoft.

Original 3D illustration of a processor with two action ports — Original 3D concept illustration by Open Source Factory. The chip and action ports explain the design; they are not a photograph of hardware or a recording of the model.

What does the model actually control?

Mage-VL-Cowork produces tool calls. A separate execution environment performs them and sends back observations. One route clicks, types and reads the next screenshot. The other runs a shell command or Python script and receives text output. The weights alone do not provide a working desktop assistant.

That division matters when a task crosses application boundaries. A workbook may be easier to edit as structured data, while a visual adjustment may still need the application’s interface. A file edit also needs a check in the program that will open it. That last handoff is where many impressive-looking agents become less convincing.

Microsoft’s reference hybrid agent makes the observation contract visible: GUI operations return a screenshot; shell operations return command output. The model chooses between those channels inside one policy. A good reader test is therefore simple: inspect both the action and what the environment returned.

A screenshot feeds one 4B policy, which chooses GUI or code before checking the result — Author-made routing diagram based on the Microsoft reference agent. “Check” describes the task’s verification step; it is not an independently measured success rate.

What does the reported 66% tell us?

The model card reports a 66.0% OSWorld-Verified headline result. Its protocol uses 361 held-out tasks, excludes Google Drive, limits each task to 50 steps and uses a 1920 × 1080 desktop. The two reported runs score 65.33% and 66.75%, averaging 66.04% before headline rounding.

Keep that whole sentence attached to the number. A different task set, step allowance, tool environment or screen setup changes the comparison. OSWorld itself distinguishes its Verified revision from older results. A leaderboard cell without its evaluation setup leaves out part of the product.

The category breakdown is more useful than treating 66% as a universal reliability estimate. Microsoft reports 87.5% on operating-system tasks, versus 62.7% on multi-application workflows. That gap is a reason to test the actual work you need done. It does not identify why any particular task failed.

Microsoft-reported OSWorld-Verified category scores, with 66.0 percent overall — Microsoft-reported results, redrawn by Open Source Factory. The overall value is the published aggregate, not the simple mean of the category bars. These are not our measurements.

There is a small precision trap in the release: its stage table prints 66.04%, while some gain statements use the rounded 66.0%. I use the headline rounding here and avoid turning that mismatch into an exact improvement claim. More importantly, the reported result is an outcome of the model and its execution setup.

What do the demonstrations show?

The official project demonstrations make the route choice tangible. In the spreadsheet example, the team reports completion in 39 seconds and six steps, using programmatic workbook edits. In the GIMP example, the agent works through the interface and uses the terminal to inspect the saved image’s properties.

Official Microsoft side-by-side spreadsheet demonstration at the 22-second mark — Official demonstration poster: Microsoft Mage Team. This image shows a selected vendor demonstration, not our test. The comparison also changes the model and tool access, so it cannot isolate the benefit of hybrid actions.

I would not turn those examples into an average speedup. Selected tasks do not tell us how often the agent chooses badly, how much recovery costs, or how long a different computer needs for inference. What they do provide is a readable hypothesis: use code where the file structure is clear, and keep the visual channel for work that depends on the interface.

Can a “4B” label tell you which GPU to buy?

No. I checked the pinned release file listing: the two BF16 safetensors shards total 9,483,668,136 bytes, about 9.48 decimal GB or 8.83 GiB. That is storage occupied by the weight files. It does not include the memory needed while handling screenshots or generating a response.

The configuration also gives enough information for a limited planning calculation. At batch one with BF16 keys and values, its 36 full-attention text layers, eight KV heads and 128-dimensional heads imply 147,456 cache bytes per occupied token. That is about 1.125 GiB at 8,192 tokens, or 4.5 GiB at 32,768 tokens.

Illustrative weight-file and text KV-cache memory totals for 8K and 32K occupied tokens — Author calculation from the pinned model configuration and file metadata. The chart excludes vision activations, temporary buffers and runtime overhead. It is not a measured memory footprint, a universal minimum, or a GPU buying recommendation.

Those estimates assume the shown precision and a conventional full text KV cache. They are not observations of a particular allocator. Quantization, offloading and a different runtime can change the footprint. The practical point is narrower: a model file that appears to fit does not establish a working inference setup.

There is also an architecture detail worth checking before downloading everything. This release declares llava_onevision2 and uses custom model code. The documented path is Transformers or SGLang with that implementation, rather than a standard Qwen-VL loader chosen just because the language backbone is Qwen-derived.

Before you run it	What the release establishes
Weights and code	Model card: Apache-2.0 weights. Repository root: MIT code license. Preserve the applicable notices.
Reference serving path	Docker launcher for the pinned SGLang environment; localhost port binding and a read-only model mount.
Request format	The reference agent renders the processor’s chat template and uses SGLang’s native `/generate` endpoint with image data.
Desktop environment	The reference agent contains GNOME and guest-workspace setup. This article does not establish native Windows desktop operation.
Our local check	Metadata, source and memory arithmetic only. No model loading, task run, speed test or measured peak memory.

What would make this worth using?

I would judge it by the saved result. For a workbook task, keep a copy of the source, state the exact cells that may change, and reopen the output in the target application. Check that the chart refers to the right range and that unrelated formulas survive. A fluent completion message cannot answer those questions.

Proposed checks for completed workbook, image and browser tasks — Proposed acceptance criteria by Open Source Factory. They are a plan for evaluation, not results from a local Mage-VL-Cowork run.

For image editing, check dimensions, transparency and whether the native document still opens. For a browser task, inspect the saved state as well as the visible confirmation. Record attempts and recovery steps, rather than counting only the final successful example. That is how a follow-up could test whether hybrid routing actually helps.

This connects with the output-contract problem in my Liquid LFM3 article and the job-first approach in the Muse, Grok Bot and Dots discussion. A useful agent needs both a way to act and an observable finish line. Mage-VL-Cowork gives us a compact, inspectable design for the first half. The release is promising enough to evaluate against a real task; its published scores do not finish that evaluation for us.

Questions before trying Mage-VL-Cowork

Is it a ready-to-install PC assistant?

The public release is a model and research/reference tooling. You still need inference, an action executor, a controlled desktop and a way to judge the result.

Does 66% mean it succeeds on 66% of my work?

No. It describes Microsoft’s stated OSWorld-Verified protocol. Your applications, documents, task definitions and execution environment may differ.

Can I load it as an ordinary Qwen vision model?

The release uses a custom llava_onevision2 architecture. Follow its documented loader and processor. This article has not verified an Ollama package or a native Windows route.

Did Open Source Factory run these tasks?

No. We inspected the public release and reference code and recalculated file size and an illustrative text-cache budget. Benchmark scores and application demonstrations are attributed to Microsoft.

Sources and production notes

Weight revision 6fbe7f91da47; repository revision d213d430de48. Original diagrams and 3D animation: Open Source Factory. The official demo poster is credited to Microsoft. The Korean archive replaces that English-language poster with translated explanation. No affiliate links.