Inco AI's Splash Runs Qwen3.8-27B Twice as Fast on Apple Silicon
Inco's open-source Splash engine runs Qwen3.8-27B at 74 tok/s on an M5 Pro, hitting 2x oMLX and 3x Ollama on Apple silicon.
- Inco released Splash, an open-source Apache-2.0 inference engine for Apple silicon.
- Runs Qwen3.8-27B at 74 tok/s on M5 Pro, 2x faster than oMLX.
- Cache reuse returns first token in 282 ms on 32K prompts, 7.3x faster than oMLX.
- Four concurrent subagents hit 3.9x aggregate decode versus next-fastest engine.
- Ships fused Metal kernels, DFlash 2 draft, and 8-bit KV cache per model.
- Install via
brew install incoai/tap/splash; needs M3+, macOS 26.4, 36 GB RAM.
Splash specializes local LLM inference for Apple silicon
Inco AI has released Splash, an Apache-2.0 server that runs selected large language models locally on recent Macs and exposes familiar HTTP APIs. Each supported model comes with 4-bit weights, a matching draft model, shape-specific Metal kernels, and a fixed memory plan. In Inco’s tests on an M5 Pro system with 48 GB of unified memory, Qwen3.8-27B generated 74 tokens per second on short prompts, versus 38 for oMLX, and reached almost four times oMLX’s aggregate throughput across four concurrent requests.
Coding agents repeatedly send growing conversation histories, reuse repository context, and fan work out to subagents. Splash targets those patterns with prefix caching and concurrent request scheduling, reducing the time spent reprocessing shared input.
Homebrew to localhost
- Chip: Apple M3 or newer
- Operating system: macOS 26.4 or later
- Memory: 36 GB of unified memory minimum, with 48 GB or more recommended
- Package manager: Homebrew
brew install incoai/tap/splash
splash serve --model incoai/Qwen3.8-27B-SplashThe first serve launch downloads the 17.4 GB model package, verifies it, checks available memory, and starts the server. The default listener binds to 127.0.0.1:8000, keeping the service on the local machine.
The quick-start path requires no configuration file. Client helpers include splash opencode, splash claude, splash codex, and splash hermes.
The benchmark lead, row by row
Inco compared Splash with oMLX, Lily, uzu, and Ollama on the same 48 GB M5 Pro. The test used coding prompts from NVIDIA’s SPEED-Bench and capped generated output at 1,024 tokens.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
