---
title: "Dynatrace Turns Mamba2 Into a Memory-Efficient Embedder That Beats Transformers"
subtitle: "A vertical chunking trick on Mamba2 keeps embedding memory flat regardless of document length, matching transformer quality on standard benchmarks."
url: "https://alphasignal.ai/news/dynatrace-turns-mamba2-into-a-memory-efficient-embedder-that-beats-transformers"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-20T12:50:15.000Z"
updated: "2026-08-21T19:52:33.507Z"
type: "Paper"
topics:
  - "Retrieval"
  - "Llms"
subtopics:
  - "Embeddings"
  - "Long Context"
upvotes: 16
source_url: "https://arxiv.org/abs/2604.18199"
comments: 0
truncated: true
---

# Dynatrace Turns Mamba2 Into a Memory-Efficient Embedder That Beats Transformers

> A vertical chunking trick on Mamba2 keeps embedding memory flat regardless of document length, matching transformer quality on standard benchmarks.

Text embeddings power search, retrieval, and clustering across modern AI systems, but the transformer models that generate them get expensive fast on long inputs. A new paper from Dynatrace Research and TU Wien argues that recurrent architectures can take over that job, offering roughly the same quality at a fraction of the memory cost when documents get long.

The authors fine-tune [Mamba2-based language models](https://arxiv.org/abs/2604.18199) into general-purpose embedders and introduce an inference schedule that keeps GPU memory constant in the input length once sequences pass a chosen chunk size. They also verify the trick generalizes to two other non-transformer families, RWKV and xLSTM.

## Why transformer embedders hit a wall

Transformer-based embedding models suffer from quadratic computational and linear memory complexity, which caps their utility on long sequences. Doubling document length roughly quadruples compute and doubles the activation memory needed to encode the input. For retrieval over 32K-token contracts, papers, or codebases, this quickly exceeds what fits on a single accelerator.

Recurrent models like Mamba2, RWKV, and xLSTM already solve the scaling problem for text generation by maintaining a fixed-size latent state. Their behavior as general-purpose embedders, where you encode a whole input in one shot rather than decoding token by token, has been largely unexplored outside a few task-specific efforts.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Linear-Time and Constant-Memory Text Embeddings Based on Recurrent Language Models](https://arxiv.org/abs/2604.18199)
- [Linear-Time and Constant-Memory Text Embeddings (HTML)](https://arxiv.org/html/2604.18199v1)
- [embed-mamba2 model collection](https://huggingface.co/collections/dynatrace-oss/embed-mamba2)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality (Mamba2)](https://arxiv.org/abs/2405.21060)