Skip to main content
Back

Achieve 67% Fewer Errors in RAG with Anthropic's Method

READ MORE

September 30, 2024 6 min read

Hey,

Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.

We identify and summarize the top 1% news, papers, models, and repos in the AI industry. 

IN TODAY'S SIGNAL

Read time: 6 min 27 sec

🎖️ Top News


📌 Encord

⚡️ Trending Signals

📌 Guidde


⚙️ Top Tutorials


🧠 How to

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

RAG

Anthropic reduces the error rate of RAGs by 67% using a simple method called  "Contextual Retrieval".

⇧ 4,800 Likes

What's New

Anthropic has developed Contextual Retrieval, a new method to enhance Retrieval-Augmented Generation (RAG) systems. This technique addresses a common problem in RAG where context is lost when encoding information, leading to retrieval failures.


The solution for that is a very simple method

  • They add important context to small text chunks before storing them, which improves accuracy later.

  • Instead of just saying “the company grew by 3%,” it includes details like which company and when, making it much easier for the system to find the right answers.


Key highlights

  • Reduces top-20-chunk retrieval failure rate by 49%

  • Combines Contextual Embeddings and Contextual BM25

  • Further improvement to 67% reduction when using reranking

  • Costs approximately $1.02 per million document tokens

  • Works across various knowledge domains and embedding models

  • Contextual Embeddings alone reduced the top-20-chunk retrieval failure rate by 35%, from 5.7% to 3.7%. When combined with Contextual BM25, the failure rate dropped by 49%, reaching 2.9%.

  • Contextual Retrieval prepends chunk-specific explanatory context to each piece of information before embedding and indexing. 

  • This preserves crucial context that might otherwise be lost when breaking documents into smaller chunks. The method employs two main techniques: Contextual Embeddings and Contextual BM25.


Technical Details of Contextual Retrieval

  • Claude generates the chunk-specific context automatically using a prompt that instructs it to provide concise, relevant information about each chunk's place within the overall document. 

  • This process leverages Anthropic's prompt caching feature to reduce costs and processing time.

  • Performance tests show significant improvements across different datasets and embedding models. 

  • Anthropic's experiments revealed that Gemini and Voyage embeddings were particularly effective with this method. 

  • They also found that using 20 chunks for retrieval provided optimal performance. 

Implementations and Cookbook

  • Integration with existing RAG systems is facilitated by Anthropic's provided cookbook, allowing developers to implement Contextual Retrieval in their own projects.

  • This resource addresses potential questions about scalability and ease of adoption.

While the method shows promise, ML engineers should consider factors such as chunk size, embedding model choice, and the potential need for domain-specific contextualizer prompts when implementing Contextual Retrieval.

Read more

How To Build An AI Agent On Your Own Data

Developers: Label more data, faster.


Accelerate data labeling workflows with custom Agents. Pre-label and pre-classify large datasets at scale, cutting time and costs for downstream labeling tasks.


A new webinar was just announced where ML experts demonstrate building custom Agents to enhance data workflows. Join now to optimize your labeling process.

Apply to attend

partner with us

TRENDING SIGNALS

AI Regulation

California Governor blocks the landmark AI safety bill SB 1047, allowing faster AI development

⇧ 8,151 Likes

AI Agent

Langchain shows-off open-source Github repo for controllable Agent for complex RAG, enhancing data-driven Q&A capabilities

⇧360 Likes

AI Podcast

Researchers unveil open-source NotebookLM alternative using Llama 3.1 405B for PDF-to-podcast conversion

⇧ 2,921 Likes

Andrej Karpathy

Andrej Karpathy's tweet goes viral showing Google's NotebookLM's deep-dive AI podcast creation novelty

⇧ 5,500 Likes

Multimodal Model

BAAI unveils state-of-the-art multimodal models trained solely with next-token prediction, surpassing SDXL and LLaVA

⇧ 600 Likes

Create how-to video guides fast and easy with AI

Guidde just released a GPT-powered tool that transforms workplace documentation. It converts repetitive explanations into visual guides quickly.


You can now create, share, and manage dynamic content with ease. The tool enhances team communication by turning routine tasks into clear, visual instructions. Guidde streamlines the process of making how-to videos, saving time and improving clarity. And their platform is free to use.

Check out Guidde

TOP TUTORIALS

AI Agent

Build an AI customer support system with OpenAI and Slack integration

⇧ 1,451 Likes

This tutorial shows you how to create a system that directs user queries to the right Slack channels. You'll see Cursor Composer in action, writing 95% of the code through voice commands. Gain hands-on experience in AI-assisted development for practical applications.
You'll understand how to integrate language models with communication platforms. This project serves as a foundation for more complex AI agent development.

Dataset creation

Generate synthetic data with Llama 3.1 for fine-tuning domain-specific language models

⇧ 630 Likes

This tutorial shows you how to generate high-quality instruction-response pairs. You'll use multiple LLMs to produce and judge responses. The process helps create domain-specific data for preference fine-tuning. You'll explore the open-source tools and Llama 3.1's new license. This technique is valuable for building specialized language models.

Finetuning Image Model

Fine-tune FLUX.1 to generate personalized images using your own photos

⇧ 943 Likes

You'll create custom models using your own photos. This tutorial covers image preparation, trigger word selection, and model training on Replicate. You'll explore web-based and API-driven image generation. It also shows how to use language models for prompt creation.

HOW TO

RAG

How to do high-accuracy RAG on PDFs with a focus on the scientific literature with PaperQA2

⇧ 1,900 Likes

PaperQA2 is a package for high-accuracy retrieval augmented generation (RAG) on scientific literature. It focuses on PDFs and text files, offering state-of-the-art implementation with document metadata-awareness in embeddings and LLM-based re-ranking. PaperQA2 supports agentic RAG, allowing language agents to refine queries and answers iteratively. It automatically fetches paper metadata, including citation and journal quality data from multiple providers.


You'll learn

  • Implementing RAG workflows

  • Document parsing with LlamaParse

  • Metadata-aware embeddings

  • LLM-based re-ranking and contextual summarization

  • Agentic RAG techniques

  • Full-text search engine setup

  • Customizing LLM models and embeddings

Step-by-step implementation

  1. Install PaperQA2:
    pip install paper-qa>=5

  2. Set up API keys:

    PaperQA2 uses an LLM to operate, so you'll need to either set an appropriate API key environment variable (i.e. export OPENAI_API_KEY=sk-...) or set up an open source LLM server

  3. Prepare your document collection:

    Gather PDFs or text files in a directory

  4. The fastest way to test PaperQA2 is via the CLI. First navigate to a directory with some papers and use the pqa cli
    $ pqa ask 'What manufacturing challenges are unique to bispecific antibodies?'

  5. Customize as needed:
    Adjust embedding models
    Change LLM settings
    Modify number of sources

Read more

Thank You.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Advertise