---
title: "LSTM Co-Inventor's NXAI Releases KronLSTM to Beat Long-Context Memory Limits"
subtitle: "A new xLSTM variant corrects one scalar instead of a whole vector per memory update, cutting interference and improving long context performance."
url: "https://alphasignal.ai/news/lstm-co-inventor-s-nxai-releases-kronlstm-to-beat-long-context-memory-limits"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-10-19T17:56:52.000Z"
updated: "2026-10-20T12:05:51.707Z"
type: "Paper"
topics:
  - "Llms"
  - "Open Source"
subtopics:
  - "Long Context"
  - "Small Models"
upvotes: 96
source_url: "https://arxiv.org/abs/2610.26834"
comments: 0
truncated: true
---

# LSTM Co-Inventor's NXAI Releases KronLSTM to Beat Long-Context Memory Limits

> A new xLSTM variant corrects one scalar instead of a whole vector per memory update, cutting interference and improving long context performance.

NXAI researchers, including LSTM co-inventor Sepp Hochreiter, have introduced the [KronLSTM paper](https://arxiv.org/abs/2610.26834), which proposes a narrower way to revise matrix memory in recurrent linear-attention models. KronLSTM adjusts the scalar coupling between a key and a value, limiting interference with other stored associations. The reported gains are strongest on long-context retrieval and state-tracking benchmarks.

* **Core change:** The Kronecker Delta Rule constrains each memory correction to one direction in matrix space.
* **Practical goal:** Preserve earlier associations when overlapping keys receive new values.
* **Reported evidence:** Higher 32K retrieval scores than Gated DeltaNet-2, plus competitive state-tracking results and comparable training throughput.
* **Current limit:** Published checkpoints reach 1.5 billion parameters and 100 billion training tokens.

## Why matrix memories collide

Standard self-attention forms pairwise interactions between tokens, giving it quadratic cost as sequence length grows. Linear attention compresses prior tokens into a fixed-size recurrent state, allowing sequence processing to scale linearly with length. That efficiency makes the quality of each memory update especially important.

Hebbian and delta updates expose the trade-off between accumulation and revision. A Hebbian update adds a key-value contribution to memory at every token, so old values can be buried by later writes but cannot be directly corrected. A delta update reads the value associated with a key, subtracts that full `d`\-dimensional vector, and writes a replacement, where

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Kronecker Delta Rule for Long Context Linear Attention](https://arxiv.org/abs/2610.26834)
- [NX-AI on Hugging Face](https://huggingface.co/NX-AI)
- [KronLSTM GitHub Repository](https://github.com/NX-AI/KronLSTM)
- [xLSTM: Extended Long Short-Term Memory](https://arxiv.org/pdf/2405.04517)
