LlamaIndex's semtools Turns Semantic Search Into a Unix Shell Command
A Rust CLI from LlamaIndex brings semantic search, document parsing, and an AI agent together as composable Unix tools you can pipe with grep and xargs.
PRO- semtools is a Rust CLI from LlamaIndex for parsing docs and semantic search from the terminal.
- Four subcommands:
parse,search,ask(agent), andworkspacefor persistent indexes. - Uses static model2vec embeddings, no GPU or Python runtime required for search.
- v3.0.0 unified all commands under one binary and added
--jsonoutput for every subcommand. - Workspaces persist embeddings in LanceDB with IVF_PQ indexing for large document collections.
- Install via
npm i -g @llamaindex/semtoolsorcargo install semtools, MIT licensed.
LlamaIndex has quietly turned semantic search into a first-class citizen of the Unix shell. Their open-source project semtools is a Rust-built command-line toolkit that parses PDFs, searches text with embeddings, and runs an agent over document collections, all with the same pipe-and-xargs ergonomics as grep. It has crossed 1.8k stars on GitHub and ships under an MIT license.
Four commands, one binary
Everything lives behind a single semtools entry point, with local-first operation for search and workspaces plus optional cloud services for parsing and question answering:
- parse converts PDFs, DOCX, PPTX and other formats to markdown via the LlamaParse API.
- search runs fast local semantic search using multilingual static embeddings.
- ask answers questions over document collections through an AI agent (defaults to OpenAI, though any OpenAI-compatible endpoint works).
- workspace persists embeddings so repeat searches on large corpora stay cheap.
Version 3.0.0 unified these subcommands under one binary and added --json output everywhere. Structured JSON means you can pipe results into jq, embed the tool in shell scripts, or hand it to a coding agent as a callable.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
