--- title: "IBM's Abstract-CoT Teaches AI to Reason With 11x Fewer Tokens" subtitle: "IBM Research's Abstract Chain-of-Thought replaces verbose reasoning traces with a compact sequence of learned tokens, cutting inference costs by up to 11.6x while matching verbalized CoT performance." url: "https://alphasignal.ai/news/ibm-s-abstract-cot-teaches-ai-to-reason-with-11x-fewer-tokens" author: "AlphaSignal Newsroom" author_url: "https://alphasignal.ai/editorial-team" company: "AlphaSignal" company_url: "https://alphasignal.ai/company/alphasignal" published: "2026-04-24T16:45:51.000Z" updated: "2026-06-09T07:57:50.425Z" type: "Paper" topics: - "Reasoning" - "Post Training" subtopics: - "Test Time Compute" - "Rlhf" - "Distillation" upvotes: 1020 source_url: "https://arxiv.org/abs/2604.22709" comments: 0 truncated: true --- # IBM's Abstract-CoT Teaches AI to Reason With 11x Fewer Tokens > IBM Research's Abstract Chain-of-Thought replaces verbose reasoning traces with a compact sequence of learned tokens, cutting inference costs by up to 11.6x while matching verbalized CoT performance. Every time a modern reasoning model thinks through a hard problem, it generates hundreds or thousands of tokens of internal monologue before producing an answer. That verbosity has a real cost: slower responses, higher compute bills, and longer training rollouts during reinforcement learning. A new paper from IBM Research proposes a different approach entirely. Instead of thinking in English, what if the model thought in a compact, learned language of its own? The paper, ["Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought"](https://arxiv.org/abs/2604.22709), introduces **Abstract Chain-of-Thought (Abstract-CoT)**, a post-training mechanism that replaces natural language reasoning traces with short sequences of reserved, non-human-readable tokens. The model still produces a final answer in plain language, but the scratchpad it uses to get there is a compact sequence of abstract symbols like ``, ``, ``, and so on. ![Comparison of verbalized chain-of-thought versus abstract chain-of-thought for the same physics problem](https://arxiv.org/html/2604.22709v2/figs/fig_cot_comparison_darker.png) ## The problem with thinking out loud Large language models increasingly rely on long, explicit chains-of-thought to solve complex, multi-step reasoning problems. Despite its effectiveness, verbalized CoT is an expensive mechanism, increasing latency and cost at inference while bloating the length of traces during reinforcement learning. There is also a deeper issue: prior works suggest that verbalized CoT can be unfaithful, while leveraging a different latent reasoning process that is not communicated. The field has explored alternatives. Continuous latent reasoning methods, like [Coconut](https://openreview.net/forum?id=tG4SgayTtk), replace CoT tokens with continuous hidden-state vectors. By internalizing multi-step inference in a model's latent space, latent CoT aims to overcome the expressive and computational bottlenecks associated with token-level chain-of-thought output. But continuous methods come with their own problems: they are harder to control, difficult to analyze, and their performance tends to lag behind verbalized CoT. Abstract-CoT sits in the middle ground: fully discrete (like text), but non-verbal (like latent reasoning). ## A new vocabulary for thinking The core idea is simple. The model produces a short sequence of tokens from a reserved abstract vocabulary in lieu of a natural language CoT, before generating a response. These tokens, named things like `` through `` and beyond, are added to the model's tokenizer as entirely new entries with randomly initialized embeddings. At inference time, the model generates a bounded sequence of these abstract tokens inside _Preview only. Full text requires an AlphaSignal Pro account._ ## Sources - [Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought](https://arxiv.org/abs/2604.22709) - [Abstract Chain-of-Thought (HTML paper)](https://arxiv.org/html/2604.22709) - [Latent Chain-of-Thought Reasoning - Emergent Mind](https://www.emergentmind.com/topics/latent-chain-of-thought-latent-cot-reasoning) - [Dolci-Think-SFT-7B Dataset (AI2)](https://huggingface.co/datasets/allenai/Dolci-Think-SFT-7B) - [Coconut: Training LLMs to Reason in a Continuous Latent Space](https://openreview.net/forum?id=tG4SgayTtk)