KoboldCpp Now Splits Giant AI Models Across Multiple Home Computers
KoboldCpp v1.118.1 adds distributed multi-GPU inference over the network, LTX2.3 video generation, and llama.cpp RPC interoperability.
Takeaways
- KoboldCpp v1.118.1 adds RPC backend support for multi-GPU inference across networked machines.
- LTX2.3 text-to-video and image-to-video generation is now integrated via a downloadable template.
- RPC protocol was realigned so KoboldCpp and llama.cpp servers and clients are interchangeable.
- Fixes include Qwen Image quality regression, multi-GPU selection bugs, and DeepSeek v4 flash prompt handling.
- Image generation can now be aborted mid-run by closing the client connection.
- Available as a single portable binary for Windows, Linux, and Apple Silicon Mac.
The single-file local LLM runner KoboldCpp just shipped a release that pushes it well past the usual chat-and-image-gen box. Version 1.118.1 from maintainer LostRuins introduces distributed GPU inference over a network, adds LTX2.3 video generation, and realigns its RPC protocol with upstream llama.cpp so servers and clients from either project can now talk to each other.
For readers new to the project, KoboldCpp is a fork of llama.cpp packaged as a portable executable with a built-in web UI, an OpenAI-compatible API, image generation, TTS, and speech-to-text. The single binary handles LLM text generation for any GGML or GGUF model, plus image generation with Stable Diffusion 1.5, SDXL, SD3, and Flux. The new release keeps that no-install ethos while expanding what a laptop, or a small cluster of them, can actually run locally.
Splitting a model across machines
The headline change is RPC backend support, which lets multiple GPUs be pooled over the network for distributed generation across several computers. You configure it from the Network tab, or with