AMD's Open-Source Lemonade Runs LLMs, Images, and Speech on Your Own PC
AMD's open-source Lemonade Server runs LLMs, image gen, and speech locally on GPU and NPU with zero cloud dependency
ProTakeaways
- Multi-modal local AI server: Lemonade runs LLMs, image gen, speech-to-text, and TTS behind a single OpenAI-compatible endpoint, 100% free and private.
- Hybrid NPU+GPU execution: On AMD Ryzen AI 300/400 hardware, Lemonade routes prefill to the XDNA2 NPU (2.3x faster time-to-first-token) and token decode to the GPU (2.4x higher throughput).
- MCP server support added (v10.8): GitHub Copilot, Claude Desktop, and Cursor can now connect directly to locally running models via the Model Context Protocol.
- NVIDIA CUDA support added (v10.7): Turing and newer GPUs now supported on Windows and Linux, though without the NPU hybrid path.
- Apache 2.0, free to use: Available on Windows, Linux, macOS, and Docker; NPU kernels (FastFlowLM) are proprietary binaries with free non-commercial use. GitHub
- Strix Halo standout: Ryzen AI MAX+ 395 with 128GB unified memory runs 70B+ models in-memory, with community benchmarks showing GPT-OSS 120B at ~50 t/s.
Lemonade is an AMD-sponsored, open-source local AI server that lets you run LLMs, image generation, speech-to-text, and text-to-speech entirely on your own hardware. Think of it as Ollama, but built from the ground up to squeeze every last drop of performance from AMD silicon, including the NPU that ships in modern Ryzen AI laptops and desktops. It has been quietly gaining traction in the AMD community, and a recent wave of updates has made it genuinely worth paying attention to.
One server, every modality
Lemonade gives you the same capabilities as cloud APIs, except 100% free and private, covering chat, coding, speech, and image generation on your own NPU and GPU. That multi-modal scope is the real pitch. Lemonade Server installs a service you can connect to hundreds of apps using standard OpenAI, Anthropic, and Ollama APIs, while Embeddable Lemonade is a portable binary you can package into your own application to give it multi-modal local AI that auto-optimizes for your user's PC.
The supported modalities in a single install:
- Text generation via
llamacpp(Vulkan, ROCm, CUDA, CPU, Metal) and AMD-specific NPU runtimes - Speech-to-text via
whispercppand Moonshine, including NPU-accelerated transcription on Windows - Text-to-speech via Kokoro
- Image generation via
stable-diffusion.cppwith ROCm, Vulkan, and CUDA backends
The hybrid NPU+GPU trick
LLM inference has two distinct phases that favor very different hardware. Prefill (prompt processing) is compute-bound: the model ingests your entire input and builds the KV cache, requiring raw matrix multiply throughput. A prompt of 1,000 tokens needs thousands of matrix operations processed in parallel, and the NPU excels here. Decode (token generation) is memory-bandwidth-bound: the model generates one token at a time, loading the entire model's weights from memory per step. The GPU wins here because it has wider memory buses.
On Ryzen AI 300/400-series hardware, Lemonade routes prompt processing through the XDNA 2 NPU and token generation through the integrated GPU (or a discrete Radeon if available). AMD's own benchmarks show the NPU delivers 2.3x faster time-to-first-token versus GPU-only inference, while GPU decode achieves 2.4x higher sustained throughput versus NPU-only decode. The hybrid mode combines both: fast startup from the NPU, sustained throughput from the GPU.