Portable-AI-USB Puts a Private Local AI on Your Keychain
Portable-AI-USB bundles Ollama, Llama 3, and AnythingLLM onto a flash drive for a fully offline, zero-trace AI that leaves no footprint on any host machine.
Takeaways
- Portable-AI-USB bundles Ollama, GGUF models, and AnythingLLM onto a USB drive for a fully offline, zero-trace AI assistant on any machine.
- Six pre-curated models available (2B to 12B), plus support for any custom HuggingFace GGUF model via a paste-and-go installer option.
- Leaves no registry keys, files, or cache on the host PC — all chats and settings stay on the drive.
- Runs on CPU only; expect 10–30 second response times. Requires at least 6–8 GB host RAM depending on model size.
- MIT licensed and free; one-time internet connection needed for setup, fully air-gapped after that. GitHub repo.
- Author also released USB-Uncensored-LLM, a follow-up with GPU acceleration (CUDA, Metal, AVX) support.
What if your AI assistant lived on your keychain? Portable-AI-USB is an open-source project that packages a full local LLM stack onto a USB flash drive. Plug it into any Windows, Mac, or Linux machine, double-click a launcher script, and you have a private AI chat interface running entirely from the drive. No cloud. No account. No trace left behind when you unplug.
The stack under the hood
The project wires together three open-source tools that have each matured significantly over the past year:
- Ollama , the inference engine. Ollama has established itself as the de facto standard for running LLMs on consumer hardware. It handles model loading, quantization, and hardware detection automatically. The installer drops a self-contained Ollama binary directly onto the USB, so nothing touches the host system's PATH or registry.
- GGUF model files , the model weights. GGUF is the universal format for local LLM inference, developed as part of the llama.cpp ecosystem. It is a single-file format that bundles model weights, tokenizer, and metadata. This makes it ideal for portable storage.
- AnythingLLM , the chat interface. AnythingLLM positions itself as an all-in-one AI application that bundles local/remote LLM chat, retrieval-augmented generation (RAG), and agent workflows while minimizing infrastructure friction. The installer forces it to write all data to the USB's
anythingllm_data/folder, not the host machine.
Six models, one drive
During setup, you pick from a curated menu. The installer downloads the selected model, imports it into Ollama, and writes an installed-models.txt manifest to the drive. You can add more models later by re-running install.bat , already-downloaded models are automatically skipped.
| Model | Size | Best for |
|---|---|---|
| NemoMix Unleashed 12B |