Modified GLM-5.3-Flash: Refusal Scores and Their Limits
MODELS & COSTS
Modified GLM-5.3-Flash: Refusal Scores and Their Limits
A lower refusal rate describes behavior on a particular test set. It does not measure every capability or make a modified checkpoint interchangeable with the original. The GLM modifications below should be compared through the teams' stated methods, serving conditions and limits on independent evidence.
In this article
The numbers OrcaRouter published are the hook. Refusal behavior collapses from 96%, 93%, 97%, and 93% down to 11%, 12%, 15%, and 18% across the four categories they measured, and XSTest — the rate of wrongly refusing harmless requests — drops from 2.4% to 0.4%. Their pitch: a stock GLM-5.3-Flash was over-refusing a lot of benign-but-flagged requests, copyright questions being the clearest real case. The uncensored build stops that, with no LoRA and no jailbreak prompt — refusal removal written straight into the model weights at the native block-FP8 precision.
The subject of the week: GLM-5.3-Flash on Hugging Face. The uncensored builds work from this exact FP8 release. (image: zai-org / huggingface.co)
Refusal changes reported in the weights
This is GLM-5.3-Flash, the model this blog caught as the anonymous "Ox Alpha" and watched Z.ai confirm — 320B total, 18B active, MIT-licensed, natively multimodal. The uncensored builds (OrcaRouter's and a parallel one, dealignai's "CRACK") remove the safety layers the same way: directly in the weights, not via a jailbreak prompt sitting on top. That's the difference that matters — a prompt is fragile, a weight edit survives.
What removing the guardrails looks like. OrcaRouter's own numbers, lower is better on every row — including XSTest, where it means the base model wrongly refuses safe inputs less often afterwards. Chart by the author, vendor-reported.
OrcaRouter ran these measurements on its own harness. Separately, dealignai reports 320/320 compliance on HarmBench-320 for its CRACK build. Both are measurements reported by the teams that made the modifications; neither establishes how the models will behave across every deployment or request.
Reported capability and serving costs
The capability side, dealignai's CRACK build: essentially unchanged MMLU-logit, 86.26% vs 86.74% on 1,026 questions. The knowledge survives removing the refusal layer. Chart by the author, vendor-reported.
This is the part that surprises people. Cutting an entire behavior out of the weights — and they don't hide the real KL-divergence from base — still leaves MMLU within half a point. That's because the safety tuning in a model like this is a thin overlay on the same knowledge. Ablate the refusal and you get the same reasoning with the brakes off. On H200 hardware it runs at native FP8 speed, 163 tok/s decode, 211 with the MTP speculative head.
| Spec | GLM-5.3-Flash uncensored FP8 |
|---|---|
| Parameters | 320B total / 18B active (MoE) |
| Quantization | Block-FP8 (e4m3) — native on Hopper |
| Context | 1M tokens |
| Vision | GLM-4.1V vision tower, working |
| Decode (H200, TP4) | 163 tok/s single-stream (211 with MTP) |
| HarmBench-320 | 320/320 complied |
| MMLU-logit | 86.26% vs 86.74% base (−0.48 pp) |
The main specs, from the dealignai CRACK card. Vendor-reported.
The branding on dealignai's release. "CRACK" is their name for permanent, weight-level uncensoring. (image: dealignai, via Hugging Face)
How this release fits earlier modifications
A frontier model's lifecycle, one week: mystery to unbounded in eight days. Chart by the author.
Every front-line open-weights release gets one of these now, usually within days. The repeated complaint underneath is real: the base models over-refuse. Copyright and other benign-but-flagged requests get refused at rates that get in the way of actually-useful use, and an uncensored build is the community's way of testing how much of the refusal layer is protective versus merely annoying. GLM's own 5.2 drew "why we don't have GLM5.2 uncensored yet?!" threads before 5.3 even landed. This one took three days.
The reported MMLU change is under half a point, alongside much larger changes in refusal rates. That comparison is limited to the published tests. It does not establish that safeguards are universally superficial, or that a future GGUF or MLX conversion will reproduce the FP8 build's behavior and speed. The cited Hopper throughput is also a reported result for that serving setup.
The sequence from anonymous Ox Alpha release to identified model to modified weights took about a week. Each stage now has a different evidence trail: Z.ai's model documentation, the modification teams' tests, and any later independent reproduction. Keeping those separate makes the results easier to check.
Related on this blog: Ox Alpha Was GLM-5.3-Flash, Trained on Chinese Chips Alone · Qwen3.8-Flash-Next: 6B Active, Qwen4's Architecture Early
Sources: OrcaRouter's uncensored-GLM-5.3-Flash announcement (X, Aug 29) and model page; dealignai/GLM-5.3-Flash-UNCENSORED-FP8 Hugging Face card (CRACK; HarmBench-320 320/320; MMLU 86.26 vs 86.74; H200 speed); zai-org/GLM-5.3-Flash HF card and r/LocalLLaMA megathread for the base specs. All uncensored-build benchmarks are the teams' own reported figures, not independent. This blog's prior posts: the Ox Alpha reveal and the GLM-5.3-Flash review.
Comments
Post a Comment