---
title: "Microsoft's BitNet b1.58 Runs a 100B AI Model on a Single CPU"
subtitle: "Microsoft's bitnet.cpp runs 100B-parameter models on a single CPU with no GPU, using ternary weights that replace float math with add/subtract ops"
url: "https://alphasignal.ai/news/microsoft-s-bitnet-b1-58-runs-a-100b-ai-model-on-a-single-cpu"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-07-27T05:52:06.000Z"
updated: "2026-08-09T13:17:47.160Z"
type: "Repo"
topics:
  - "Open Source"
  - "Gpus"
subtopics:
  - "Quantization"
  - "Edge Deployment"
upvotes: 357
source_url: "https://github.com/microsoft/BitNet"
comments: 0
truncated: true
---

# Microsoft's BitNet b1.58 Runs a 100B AI Model on a Single CPU

> Microsoft's bitnet.cpp runs 100B-parameter models on a single CPU with no GPU, using ternary weights that replace float math with add/subtract ops

[bitnet.cpp](https://github.com/microsoft/BitNet) is Microsoft's official inference framework for 1-bit LLMs. It runs a 100-billion-parameter language model on a single consumer CPU at 5–7 tokens per second, with no GPU, no cloud, and no exotic hardware required. The mechanism is a fundamentally different way of representing model weights: expensive floating-point multiplication replaced by cheap integer addition.

Microsoft released bitnet.cpp as an open-source project under the MIT license, enabling fast and lossless inference of 1.58-bit models on CPU and GPU hardware, with NPU support planned. The repo has crossed 39,000 GitHub stars and draws steady attention from both researchers and engineers.

## The math behind it

Standard LLMs store weights as 16-bit or 32-bit floating-point numbers. Every forward pass requires billions of floating-point multiplications, which is precisely why GPUs dominate inference. BitNet constrains every weight in the network to one of three values: -1, 0, or +1\. This is ternary quantization. Encoding three distinct values requires log₂(3) = 1.58 bits, hence the "1.58-bit" label.

The consequences are significant. Multiplying by -1, 0, or +1 reduces to a sign flip, a zero, or a pass-through. Matrix multiplication, the most expensive operation in transformer inference, collapses into integer additions and subtractions that any modern CPU handles efficiently. Memory footprint shrinks by over 90% compared to FP16, with surprisingly little quality loss.

## Trained ternary, not squeezed down afterward

Post-training quantization (PTQ) methods like GPTQ or AWQ take a trained full-precision model and compress it after the fact, which typically degrades quality. BitNet works differently: all trainable weights are constrained to {-1, 0, +1} from the start of training, guided by 16-bit "shadow weights" that are quantized during each forward pass. A straight-through estimator handles gradient flow through the discrete values. Activations are also quantized within a fixed range. The model learns to operate within the constraint rather than being forced into it retroactively.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [microsoft/BitNet — GitHub](https://github.com/microsoft/BitNet)
- [BitNet b1.58 2B4T Technical Report — arXiv](https://arxiv.org/html/2504.12285v1)
- [microsoft/BitNet-b1.58-2B-4T — Hugging Face](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T)
- [Microsoft BitNet: Run 100B Parameter LLMs on Your CPU — emelia.io](https://emelia.io/hub/bitnet-1bit-llm-cpu-inference)
- [BitNet b1.58: Efficient Edge Inference for Ternary LLMs — arXiv](https://arxiv.org/pdf/2502.11880)