---
title: "Qwen3.8-27B Hits 417 tok/s on a Single RTX 3090 With 150k Context"
subtitle: "A community project squeezes Qwen3.8-27B onto a 24GB gaming card with vLLM, hitting 417 tok/s batched or 82 tok/s single-user at 150k context."
url: "https://alphasignal.ai/news/qwen3-8-27b-hits-417-tok-s-on-a-single-rtx-3090-with-150k-context"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-08-27T08:51:41.000Z"
updated: "2026-08-28T06:49:00.936Z"
type: "Repo"
topics:
  - "Infra"
  - "Gpus"
subtopics:
  - "Inference Optimization"
  - "Model Serving"
  - "Quantization"
upvotes: 961
source_url: "https://github.com/syv-ai/qwen38-27b-rtx3090"
comments: 0
truncated: true
---

# Qwen3.8-27B Hits 417 tok/s on a Single RTX 3090 With 150k Context

> A community project squeezes Qwen3.8-27B onto a 24GB gaming card with vLLM, hitting 417 tok/s batched or 82 tok/s single-user at 150k context.

Running a 27B parameter model on a single consumer GPU used to mean choosing between crippling context length and painful token rates. A new open source project called [qwen38-27b-rtx3090](https://github.com/syv-ai/qwen38-27b-rtx3090) discards that tradeoff with a tightly tuned vLLM setup that fits [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) into 24 GB of VRAM while serving a 150k token context window.

The repo bundles two ready-made configurations, a pair of requantization scripts, a small vLLM patch, and a set of benchmark harnesses. It targets a 24 GB Ampere or newer NVIDIA card, a recent driver, Python 3.12, and about 40 GB of disk. Everything ships under Apache 2.0 and is available today from the GitHub repository, with the underlying [W4A16 AutoRound weights](https://huggingface.co/dbirks/Qwen3.8-27B-W4A16-AutoRound) pulled from Hugging Face.

## Two modes, one install

The setup ships as a single install with a mode toggle. Batch mode targets API backends, pipelines, and many concurrent requests. Single-user mode is tuned for one or a few people chatting. The crossover sits around eight concurrent users: below that, speculation wins; above, plain batching pulls ahead.

The headline numbers, all measured with `vllm bench serve` on an RTX 3090 held at a conservative 250 W power limit:

* Batch mode: 417 tok/s aggregate at 64 concurrent, with 672 tok/s peak
* Single-user mode: 82 tok/s single-stream on 1,024-token generations
* Prefill: about 1,210 tok/s at 1k inputs, degrading only to 795 tok/s at 100k
* Median TPOT of 21.4 ms for single-concurrent 256/256 workloads

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [syv-ai/qwen38-27b-rtx3090 GitHub repository](https://github.com/syv-ai/qwen38-27b-rtx3090)
- [Batch mode benchmarks and knobs](https://github.com/syv-ai/qwen38-27b-rtx3090/blob/main/batch/README.md)
- [Qwen3.8-27B on Hugging Face](https://huggingface.co/Qwen/Qwen3.8-27B)
- [dbirks/Qwen3.8-27B-W4A16-AutoRound](https://huggingface.co/dbirks/Qwen3.8-27B-W4A16-AutoRound)