HKU's RAG-Anything Hits 20K Stars Letting AI Read Charts and Tables
RAG-Anything is an open-source, graph-powered RAG framework from HKUDS that processes text, images, tables, and equations inside a single unified pipeline — no more stitching together a dozen specialized tools.
PRO- What it is: RAG-Anything is a free, MIT-licensed Python framework for multimodal RAG over PDFs, Office docs, images, and more.
- Core innovation: Extends LightRAG's knowledge graph to treat text, images, tables, and equations as first-class graph entities with cross-modal relationships.
- Three query modes: Pure text, VLM-enhanced (auto-analyzes images in retrieved context), and multimodal queries (pass tables/equations directly in the query).
- Multiple parsers: Supports MinerU, Docling, and PaddleOCR; Office documents require LibreOffice installed separately.
- Best for: Academic papers, financial reports, technical documentation — any corpus where critical information lives in figures, tables, or equations.
- Caveats: Heavier compute than plain vector RAG; LLMs of 32B+ recommended for graph extraction; 99 open GitHub issues signal active but rough edges.
RAG-Anything is a multimodal RAG framework from the Hong Kong University Data Science lab that does something deceptively simple: it lets you ask questions about a document and actually get answers that account for the figures, tables, equations, and diagrams inside it , not just the text. With nearly 20,000 GitHub stars and an MIT license, it has become one of the most-watched open-source RAG projects right now.
Building a multimodal RAG system used to mean stitching together a dozen specialized tools , one for OCR, one for tables, one for math formulas, one for embeddings, one for search. Traditional RAG pipelines were designed for text, and once documents started including images, tables, equations, and charts, the toolchain quickly became messy and unmanageable. RAG-Anything replaces that mess with a single coherent framework.
The problem it's solving
Existing RAG frameworks focus exclusively on text-only knowledge while neglecting the rich multimodal information present in real-world documents. This limitation fundamentally misaligns with how information exists in authentic environments. Think about a financial report, a research paper, or a technical spec sheet: the most important data is often locked inside a chart or a table, not the prose.
This textual assumption forces existing RAG systems to either discard non-textual information entirely or flatten complex multimodal content into inadequate textual approximations. RAG-Anything treats every modality as a first-class citizen instead.
Built on a graph, not just vectors
The key architectural decision here is the choice of foundation. RAG-Anything is built on top of LightRAG, a graph-enhanced retrieval system from the same lab. Understanding LightRAG is key to understanding why RAG-Anything works the way it does.
LightRAG is a RAG system that builds knowledge graphs from your documents. Unlike classical RAG systems that rely solely on vector similarity search, LightRAG extracts entities and relationships from documents to create a structured knowledge graph, then uses both the graph and vector search for retrieval. The graph layer lets the system understand
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
