club-3090 Runs Claude-Competitive Qwen3.6-27B on a $800 Consumer GPU
club-3090 is a community repo shipping battle-tested Docker configs for running Qwen3.6-27B on a single RTX 3090 — with 85+ TPS, 125K context, vision, and tool-calling, all for free
Takeaways
- club-3090 is an open-source repo shipping validated Docker configs for running Qwen3.6-27B on 1× or 2× RTX 3090 GPUs. GitHub
- Achieves 85 TPS sustained / 106 TPS peak with 125K context, vision, tool-calling, and MTP speculative decoding on a single 24 GB card.
- Requires four layered fixes: Lorbus INT4 quant (BF16 MTP head), Genesis patches, a custom CUDA graph fix, and MTP n=3 tuning — none of which work without the others.
- Community benchmarks show thinking mode OFF outperforms thinking ON for local coding (LiveCodeBench: 64% → 90%) due to runaway reasoning loops under Q4 quantization.
- Supports vLLM, llama.cpp, and ik_llama engines; SGLang is blocked on Ampere pending an upstream Marlin fix. Now also covers Qwen3.6-35B, Gemma 4 26B, and Gemma 4 31B.
- Free, Apache 2.0, available now — requires Linux, Docker + NVIDIA runtime, driver 580+, and ~30 GB disk per model.
club-3090 is a community-maintained repository that answers one very specific question: how do you actually run a frontier-class 27B model on a consumer GPU, without it crashing, producing gibberish, or running at 10 tokens per second? The answer turns out to involve four separate patches, a carefully chosen quantization, and a surprising amount of upstream bug-fixing , all of which the repo packages into a single docker compose up.
The timing matters. A used RTX 3090 at $700–850 can now run Qwen3.6-27B, a model Qwen claims scores 77.2 on SWE-bench Verified, 59.3 on Terminal-Bench 2.0, and 83.9 on LiveCodeBench v6. That is not a hobbyist toy , it is a model competitive with Claude Sonnet 4.5 on agentic coding benchmarks, running on hardware you can buy used for less than a MacBook Air.
What the repo actually ships
club-3090 provides community recipes for serving LLMs on RTX 3090, 4090, and 5090 CUDA GPUs, with a multi-engine approach covering vLLM, llama.cpp, and ik_llama, and currently ships curated configs for Qwen3.6-27B, Qwen3.6-35B, Gemma 4 26B, and Gemma 4 31B for both 1× and 2× card setups.
The core value is not just configs , it is the opinionated decisions baked into them:
- Drop-in OpenAI-compatible API , point any OpenAI SDK at
localhost:8020and it just works - Vision, tool-calling, streaming, and thinking mode all enabled by default
- MTP speculative decoding (n=3) for a ~30–40% throughput boost on code prompts
- TurboQuant KV cache compression to push context from 8K to 125K on a single card
- Dual-card configs for 262K context and higher concurrency
The repo separates engine docs (which apply across all models) from model-specific directories, so vLLM configuration is documented once regardless of whether you are serving Qwen, GLM, or Llama. Each model lives under models/<name>/ with its own quants, quirks, and engine-specific recipes, and adding a new model means adding a new subdirectory with the same internal pattern.
The engineering problem nobody warned you about
Qwen3.6-27B looks straightforward on paper: 27B parameters, fits in INT4 at ~17 GB, comfortably under the 3090's 24 GB ceiling. The model uses a hybrid attention architecture , interleaved DeltaNet (Gated Linear Attention with a recurrent state) and standard softmax attention layers , plus a native Multi-Token Prediction (MTP) head for speculative decoding. That architecture is what makes it fast in theory and painful in practice.