---
title: "Alibaba's Qwen3.5 Runs 5x Faster on Device With a Clever Math Trick"
subtitle: "A MatMul-only matrix inversion trick delivers 5x kernel speedup for Gated DeltaNet linear attention on NPUs, without sacrificing accuracy"
url: "https://alphasignal.ai/news/alibaba-s-qwen3-5-runs-5x-faster-on-device-with-a-clever-math-trick"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-04T00:00:00.000Z"
updated: "2026-06-15T01:12:13.934Z"
type: "Paper"
topics:
  - "Infra"
  - "Gpus"
subtopics:
  - "Inference Optimization"
  - "Quantization"
upvotes: 12
source_url: "https://arxiv.org/abs/2606.06034"
comments: 0
truncated: true
---

# Alibaba's Qwen3.5 Runs 5x Faster on Device With a Clever Math Trick

> A MatMul-only matrix inversion trick delivers 5x kernel speedup for Gated DeltaNet linear attention on NPUs, without sacrificing accuracy

Linear attention is having a moment. [Qwen3.5](https://qwen.ai/blog?id=qwen3.5), Alibaba's latest flagship model family, bets heavily on it: roughly 75% of the model's layers use Gated DeltaNet, a linear attention mechanism, while the remaining 25% use conventional full attention -- a 3:1 ratio of linear to quadratic layers. The appeal is obvious. Instead of maintaining an ever-growing attention map, each Gated DeltaNet layer keeps a fixed-size state matrix with dimensions proportional to the head dimension squared, independent of sequence length. New tokens update this state incrementally, and the output for each token is produced by querying the current state -- making the cost linear in sequence length.

But there's a hidden cost buried inside this elegant design. A new paper from researchers at Luoming Zhang et al. targets one of the nastiest remaining bottlenecks: **matrix inversion inside chunk-wise parallel linear attention**, and proposes a surprisingly clean fix that delivers up to a 5x kernel-level speedup.

## The hidden bottleneck nobody talks about

To understand the problem, you need to know how Gated DeltaNet actually runs during training and inference. The sequence is split into chunks of size C. Within a chunk, the recurrence is computed exactly. The state is then passed between chunks. This chunkwise approach is what makes linear attention hardware-friendly -- it maps well to the batched matrix multiplications (GEMMs) that GPUs and NPUs are optimized for.

The problem is that computing the intra-chunk recurrence exactly requires inverting a strictly lower-triangular matrix -- a matrix where all the action is below the diagonal. The standard way to do this is **forward substitution**, a sequential algorithm where each row depends on the row before it. In AMG smoothers, polynomial Gauss-Seidel and ILU can be formulated via truncated Neumann sums, transforming triangular solves to batched matrix-vector products and providing dramatic speedups on parallel hardware -- but the standard approach in linear attention kernels hasn't made this leap. The result is a kernel that runs mostly sequentially on hardware that is built for parallelism.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [When Good Enough Is Optimal: Multiplication-Only Matrix Inversion Approximation for Quantized Gated DeltaNet](https://arxiv.org/abs/2606.06034)
- [Qwen3.5: Towards Native Multimodal Agents](https://qwen.ai/blog?id=qwen3.5)
- [Gated DeltaNet: The Surgical Eraser Solving Linear Attention's Memory Problem](https://pub.towardsai.net/gated-deltanet-the-surgical-eraser-solving-linear-attentions-memory-problem-1e50ca3e42ab)
- [Qwen3.5: Nobody Agrees on Attention Anymore](https://huggingface.co/blog/mlabonne/qwen35)
- [Performance and Efficiency Gains of NPU-Based Servers over GPUs for AI Model Inference](https://www.mdpi.com/2079-8954/13/9/797)