Advanced RAG & Multimodal

01. Query Transformation

Users rarely ask questions in the way that is most effective for retrieval. A user might ask "Why is my app slow?" when the actual documents discuss "performance optimization," "latency reduction," and "caching strategies." Query transformation bridges this gap by rewriting, expanding, or decomposing the user's query before it hits the vector store.

What This Means for Practitioners

Query transformation technique comparison:

Multi-query is always a good default -- low cost, high impact:

from openai import OpenAI
import json

client = OpenAI()

def generate_multi_queries(question: str, n: int = 3) -> list[str]:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{
            "role": "system",
            "content": f"Generate {n} alternative versions of the given question "
                       f"for document retrieval. Use different keywords and phrasing. "
                       f"Return as JSON array under key 'queries'."
        }, {"role": "user", "content": question}],
        temperature=0.7,
        response_format={"type": "json_object"}
    )
    return json.loads(response.choices[0].message.content).get("queries", [question])

HyDE works because answers look more like documents than questions do. Instead of embedding the question, generate a hypothetical answer first and embed that:

def generate_hyde(question: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{
            "role": "system",
            "content": "Write a short paragraph that would appear in a technical "
                       "document answering the given question. Write as factual "
                       "content, not as a response."
        }, {"role": "user", "content": question}],
        temperature=0.5, max_tokens=200
    )
    return response.choices[0].message.content

Query decomposition handles multi-hop questions that a single retrieval would fail on. "How does our company's RAG system compare to industry standard for financial documents?" becomes two retrievals: (1) "How does our RAG system process financial documents?" and (2) "What is the industry standard for financial document RAG?"

02. Parent-Child Chunks

The fundamental tension in chunking is between retrieval precision and context completeness. Small chunks (256 tokens) retrieve with high precision but lack surrounding context. Large chunks (2048 tokens) provide context but dilute the retrieval signal with irrelevant text. Parent-child chunking resolves this by using a two-level hierarchy: small "child" chunks are used for retrieval (high precision), but when a child matches, the system returns its larger "parent" chunk (rich context).

What This Means for Practitioners

When to use parent-child chunks:

• Users complain that answers are incomplete or missing context
• Your documents have clear section structure (manuals, contracts, guides)
• Small chunks retrieve precisely but the LLM cannot generate good answers from fragments

Size guidelines: Parent chunks: 1500-2500 tokens (full sections). Child chunks: 200-500 tokens (focused passages). Typical ratio: 3-6 children per parent.

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.storage import InMemoryStore
from langchain.retrievers import ParentDocumentRetriever
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings

parent_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=200)
child_splitter = RecursiveCharacterTextSplitter(chunk_size=400, chunk_overlap=50)

retriever = ParentDocumentRetriever(
    vectorstore=Chroma(
        collection_name="parent_child",
        embedding_function=OpenAIEmbeddings(model="text-embedding-3-small"),
    ),
    docstore=InMemoryStore(),
    child_splitter=child_splitter,
    parent_splitter=parent_splitter,
)

# Index: automatically creates parent + child chunks
retriever.add_documents(docs)

# Retrieve: searches child chunks, returns parent chunks
results = retriever.invoke("How does PagedAttention manage GPU memory?")

Sentence window retrieval is a lighter-weight alternative. Each individual sentence is embedded for maximum precision. When a sentence matches, the system returns a window of N sentences before and after. This works well for dense, information-rich content like legal contracts and medical guidelines.

03. Graph RAG

Standard RAG treats each document chunk as an independent island. There is no understanding of how concepts relate across chunks. Graph RAG builds a knowledge graph alongside the vector index -- a network of entities connected by relationships ("uses," "depends on," "is part of"). When a user asks a question, the system traverses these relationships to find connected information that simple vector similarity would miss.

Graph RAG is particularly effective for global questions that span many documents: "What are all the dependencies of our payment processing module?" or "How do these technologies relate to each other?" Standard vector search excels at local questions where the answer is in one or two chunks. Graph traversal aggregates information across the entire corpus.

What This Means for Practitioners

When Graph RAG adds value vs. when it does not:

Use Graph RAG alongside vector RAG, not instead of it. Run both in parallel and merge results. Vector search handles most queries well. Graph traversal adds value for multi-hop, relational, and global queries.

from openai import OpenAI
import networkx as nx
import json

client = OpenAI()
graph = nx.DiGraph()

def extract_graph(chunk: str) -> dict:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": f"""Extract entities and relationships.
Return JSON with:
- "entities": [{{"name": "...", "type": "concept|technology|process"}}]
- "relationships": [{{"source": "...", "target": "...", "relation": "..."}}]

Text: {chunk}"""}],
        response_format={"type": "json_object"}, temperature=0.0
    )
    return json.loads(response.choices[0].message.content)

def graph_retrieve(query: str, hops: int = 2) -> list[str]:
    query_data = extract_graph(query)
    seed_entities = [e["name"] for e in query_data.get("entities", [])]
    matched = [n for n in graph.nodes if any(
        e.lower() in n.lower() for e in seed_entities)]

    related = set(matched)
    for _ in range(hops):
        neighbors = set()
        for node in related:
            if node in graph:
                neighbors.update(graph.successors(node))
                neighbors.update(graph.predecessors(node))
        related.update(neighbors)

    return [graph.nodes[n].get("chunks", []) for n in related if n in graph.nodes]

04. Multimodal RAG

Real-world documents contain tables, charts, diagrams, and photographs that carry critical information. A financial report's value is in its tables as much as its text. Multimodal RAG extends the pipeline to handle these non-text elements.

What This Means for Practitioners

Two approaches to multimodal RAG:

Extraction-based is simpler and works with existing infrastructure:

import base64
from openai import OpenAI
from pathlib import Path

client = OpenAI()

def describe_image(image_path: str) -> str:
    b64 = base64.b64encode(Path(image_path).read_bytes()).decode()
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": [
            {"type": "text", "text": "Describe this image in detail for document retrieval. "
                                     "Include all visible text, data, and what it represents."},
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}}
        ]}],
        max_tokens=1000
    )
    return response.choices[0].message.content

Table extraction deserves special attention. Tables are the most common non-text element in enterprise documents and contain some of the most important information. Convert to markdown format, which LLMs understand well:

import pdfplumber

def extract_tables_as_markdown(pdf_path: str) -> list[str]:
    tables_md = []
    with pdfplumber.open(pdf_path) as pdf:
        for page_num, page in enumerate(pdf.pages, 1):
            for table in page.extract_tables():
                if not table:
                    continue
                headers = table[0]
                md = "| " + " | ".join(str(h) for h in headers) + " |\n"
                md += "| " + " | ".join("---" for _ in headers) + " |\n"
                for row in table[1:]:
                    md += "| " + " | ".join(str(c) for c in row) + " |\n"
                tables_md.append(f"[Table from page {page_num}]\n{md}")
    return tables_md

05. Self-Corrective RAG

Figure 2 — Self-corrective RAG loop: retrieve, grade, rewrite or generate

Standard RAG is a one-shot pipeline: retrieve, generate, done. If retrieval returns irrelevant documents or generation is unfaithful to context, there is no mechanism to detect or fix the problem. Self-corrective RAG adds feedback loops that check quality and retry when results are insufficient.

The most practical framework is CRAG (Corrective RAG): after retrieving documents, use an LLM to judge whether they are relevant. If relevant, proceed normally. If ambiguous, supplement with a web search. If irrelevant, discard and fall back to a different retrieval strategy. This prevents the common failure where the LLM generates an answer based on tangentially related but unhelpful context.

What This Means for Practitioners

When to add self-corrective loops:

• High-stakes domains where wrong answers have real consequences (medical, legal, financial)
• Users report confidently wrong answers
• Retrieval quality is inconsistent across query types

When basic RAG is sufficient:

• Low-stakes internal Q&A
• Narrow, well-indexed document collections
• Latency requirements under 2 seconds

CRAG implementation with LangGraph:

from langgraph.graph import StateGraph, END
from typing import TypedDict

class RAGState(TypedDict):
    question: str
    documents: list[str]
    generation: str
    retries: int

def retrieve(state: RAGState) -> RAGState:
    docs = retriever.search(state["question"], k=5)
    return {**state, "documents": [d["text"] for d in docs]}

def grade_documents(state: RAGState) -> RAGState:
    relevant = []
    for doc in state["documents"]:
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user",
                "content": f"Is this document relevant to: {state['question']}?\n"
                           f"Document: {doc[:500]}\nAnswer yes or no."}],
            temperature=0.0, max_tokens=3
        )
        if "yes" in response.choices[0].message.content.lower():
            relevant.append(doc)
    return {**state, "documents": relevant}

def should_rewrite(state: RAGState) -> str:
    if len(state["documents"]) < 2 and state["retries"] < 2:
        return "rewrite"
    return "generate"

def rewrite_query(state: RAGState) -> RAGState:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user",
            "content": f"This question didn't retrieve good results. "
                       f"Rewrite with different keywords.\nOriginal: {state['question']}"}],
        temperature=0.7
    )
    return {**state, "question": response.choices[0].message.content.strip(),
            "retries": state["retries"] + 1}

def generate(state: RAGState) -> RAGState:
    context = "\n\n".join(state["documents"])
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "Answer based only on the provided context."},
            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {state['question']}"}
        ], temperature=0.1
    )
    return {**state, "generation": response.choices[0].message.content}

workflow = StateGraph(RAGState)
workflow.add_node("retrieve", retrieve)
workflow.add_node("grade", grade_documents)
workflow.add_node("rewrite", rewrite_query)
workflow.add_node("generate", generate)
workflow.set_entry_point("retrieve")
workflow.add_edge("retrieve", "grade")
workflow.add_conditional_edges("grade", should_rewrite,
    {"rewrite": "rewrite", "generate": "generate"})
workflow.add_edge("rewrite", "retrieve")
workflow.add_edge("generate", END)
app = workflow.compile()

06. Production Upgrade Path

The most important production principle is: start simple, add complexity only when measurement justifies it.

What This Means for Practitioners

Incremental upgrade path for RAG systems:

Each upgrade should be A/B tested against the previous baseline. Adding every technique at once makes it impossible to measure which one actually improved results. Stage 2+3 (hybrid search + reranking) covers 80% of use cases and should be your first upgrades.

Interview Ready

How to Explain This in 2 Minutes

Likely Interview Questions

Common Mistakes

• Adding every advanced technique at once. Start with hybrid search + reranking (highest impact). Measure, then layer on query expansion and CRAG incrementally. Each addition should be A/B tested.
• Using query expansion without deduplication. Multi-query retrieval produces duplicate chunks. Always apply Reciprocal Rank Fusion and deduplicate by chunk ID.
• Treating reranker scores as absolute confidence. Cross-encoder scores are relative, not calibrated probabilities. Always set a minimum relevance threshold calibrated on your evaluation set.

Previous

07 · RAG Systems

Next

09 · Agents