AI & LLM

Updated for 2026

Retrieval-Augmented Generation (RAG) Architecture Cheatsheet

Advanced RAG techniques: hierarchical chunking, query translation, hybrid retrieval, Cross-Encoder reranking, metadata filtering, and RAG evaluation.

Target Version Compatibility

Interactive Skill Mastery

Mark commands as learned to build your customized reference tracker. Retained locally in this browser.

Level:Novice

Command Mastery Progress0 of 16 Mastered (0%)

Chunking & Loader

RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)

BeginnerBasics

Splits documents recursively by characters (\n\n, \n, space) to maintain semantically cohesive paragraph-sized chunks.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

ParentDocumentRetriever(docstore=store, vectorstore=vstore, child_splitter=child, parent_splitter=parent)

BeginnerBasics

Splits documents into small child chunks for vector search, but returns the larger parent document context to the LLM.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

ParentDocumentRetriever(docstore=store, vectorstore=vstore, child_splitter=child, parent_splitter=parent)

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

SemanticChunker(OpenAIEmbeddings(model='text-embedding-3-small'), breakpoint_threshold_type='percentile')

BeginnerBasics

Splits text dynamically based on similarity threshold changes in the embedding space, ensuring each chunk contains a single semantic concept.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

SemanticChunker(OpenAIEmbeddings(model='text-embedding-3-small'), breakpoint_threshold_type='percentile')

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

UnstructuredPDFLoader('report.pdf', mode='elements')

BeginnerBasics

Extracts layout-aware structural elements (titles, narrative, lists, tables) from complex PDFs instead of doing raw text dumping.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

UnstructuredPDFLoader('report.pdf', mode='elements')

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

Query Translation

MultiQueryRetriever.from_llm(retriever=vector_retriever, llm=chat_model)

BeginnerBasics

Uses an LLM to generate multiple variations of a user query, retrieving document candidates for all generated variants.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

MultiQueryRetriever.from_llm(retriever=vector_retriever, llm=chat_model)

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

HyDEConfig(llm=chat, embedding_model=emb_model)

BeginnerBasics

Generates a hypothetical document answering the user query, and uses that vector for similarity search to reduce the semantic gap.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

HyDEConfig(llm=chat, embedding_model=emb_model)

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

QueryRewriter(llm=chat)

BeginnerBasics

Reformulates vague or conversational query strings into precise, standalone search queries incorporating prior chat history.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

QueryRewriter(llm=chat)

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

Retrieval & Filter

vectorstore.similarity_search(query, filter={"category": "finance", "year": {"$gte": 2024}})

IntermediateAdvanced

Executes a semantic similarity search constrained by structured metadata filters for exact namespace matches.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

vectorstore.similarity_search(query, filter={"category": "finance", "year": {"$gte": 2024}})

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

vectorstore.as_retriever(search_type='mmr', search_kwargs={'k': 5, 'fetch_k': 20})

IntermediateTeam Workflow

Applies Maximal Marginal Relevance (MMR) to balance semantic query relevance with diversity, eliminating redundant information.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

vectorstore.as_retriever(search_type='mmr', search_kwargs={'k': 5, 'fetch_k': 20})

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

EnsembleRetriever(retrievers=[bm25_retriever, vector_retriever], weights=[0.3, 0.7])

BeginnerBasics

Combines sparse (BM25 lexical search) and dense (vector embedding similarity) retrieval techniques with weighted Reciprocal Rank Fusion.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

EnsembleRetriever(retrievers=[bm25_retriever, vector_retriever], weights=[0.3, 0.7])

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

ContextualCompressionRetriever(base_compressor=llm_filter, base_retriever=retriever)

AdvancedPerformance

Passes retrieved documents through an LLM to dynamically extract only the sentence spans directly relevant to the user's question.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

ContextualCompressionRetriever(base_compressor=llm_filter, base_retriever=retriever)

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

Reranking

CohereRerank(client=cohere_client, model='rerank-english-v3.0', top_n=3)

BeginnerBasics

Applies a deep learning Cross-Encoder model to compute high-accuracy relevance scores on retrieved candidate documents.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

CohereRerank(client=cohere_client, model='rerank-english-v3.0', top_n=3)

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

CrossEncoderReranker(model='BAAI/bge-reranker-v2-m3', top_n=5)

BeginnerBasics

Performs local, high-precision semantic re-scoring using an open-source Cross-Encoder model before returning context to the generator.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

CrossEncoderReranker(model='BAAI/bge-reranker-v2-m3', top_n=5)

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

Evaluation & Guardrails

evaluate(dataset, metrics=[faithfulness, answer_relevance, context_recall])

BeginnerBasics

Runs automated RAGAS evaluation metrics to assess generation quality, factual consistency, and retrieval relevance.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

evaluate(dataset, metrics=[faithfulness, answer_relevance, context_recall])

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

llama_guard.predict('User: {query} \n Assistant: {response}')

BeginnerBasics

Intercepts inputs and outputs to classify content safety and policy compliance using LlamaGuard classification templates.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

llama_guard.predict('User: {query} \n Assistant: {response}')

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

NeMoGuardrails(config=rails_config)

BeginnerBasics

Integrates runtime conversational rails to prevent prompt injections, off-topic chats, and hallucinated generator outputs.

When to Use

When building intelligence systems that augment large language models with dynamic, external, up-to-date knowledge bases.

Common Mistakes

Using giant fixed-size chunk sizes without overlaps, which cuts off cohesive contextual descriptions or fragments vital information.

Shortcut / Pro-Tip

Combine dense semantic retrieval with sparse keyword lookups (BM25) using RRF (Reciprocal Rank Fusion) for perfect accuracy.

Example

NeMoGuardrails(config=rails_config)

Output Example

Console / Terminal

Knowledge base queried, documents retrieved, reranked, and injected into model context successfully.

RAG Systems Best Practices

1Optimize with Parent-Document Retrieval

Never feed giant, raw, unfiltered documents directly. Split documents into small child passages for precise vector indexing, but retrieve the larger, cohesive parent paragraphs for the LLM to preserve complete context.

2Incorporate Hybrid (Dense + Sparse) Search

Execute semantic vector matches (Dense) alongside exact token/keyword queries (Sparse BM25) and merge them using Reciprocal Rank Fusion (RRF) to secure maximum recall coverage.

3Apply High-Performance Cross-Encoder Reranking

Introduce a fast, dedicated Cross-Encoder model (like Cohere Rerank or BGE-Reranker) to score and prioritize top documents prior to LLM injection, mitigating context distraction.

4Implement Dynamic Query Translation

Leverage automated LLM sub-query decomposition or hypothetical document embeddings (HyDE) to expand short, abstract user inputs into descriptive query vectors.

5Validate Continuously with Automated RAGAS Evaluation

Rigorously audit retrieval quality, factual consistency (faithfulness), and model hallucination rates using automated evaluation frameworks on curated test sets.

Common RAG Systems Errors & Solutions

Error

Model hallucinates or fails to mention target facts

Solution

The context window was likely flooded with irrelevant, unranked noise. Shorten overall chunk lengths, improve overlap parameters, and employ a cross-encoder reranker to keep source inputs clean.

Error

Poor search recall on acronyms, numbers, and SKU codes

Solution

Pure dense vector embeddings struggle with exact token matches. Implement a hybrid dense-sparse retriever to let keyword search (BM25) handle specialized tokens.

Error

Lost in the Middle effect causing missing facts

Solution

LLMs prioritize information placed at the very start and very end of their input context window. Rearrange retrieved paragraphs to place high-relevance chunks at the edges.

Error

Context leakage and permission cross-contamination

Solution

Enforce strict document security by attaching user-level, tenant-level, or role-based metadata to vector records and applying pre-filtering during query executions.

Error

Reranking phase introducing severe API response latencies

Solution

Reranking large candidate lists (e.g., 100+ chunks) using deep learning models is CPU/GPU intensive. Restrict initial semantic search yields to the top 20-30 chunks before reranking.

Common RAG Systems Interview Questions

Q1What is Retrieval-Augmented Generation (RAG) and why is it preferred over fine-tuning?

RAG is an architectural pattern that enhances LLM responses by retrieving relevant context from external databases at query time and injecting it directly into the prompt. It is preferred over fine-tuning for dynamic knowledge because it allows real-time updates without retraining costs, supports strict source attribution, and enforces row-level security boundaries.

Q2What is the N+1 context problem in RAG, and how do you resolve it?

The N+1 problem (context bloat) occurs when multiple similar document chunks are retrieved, repeating redundant definitions and wasting context tokens. It is resolved by employing hierarchical chunking, sentence-window retrieval, or deduplication filters during post-retrieval processing.

Q3Explain the difference between Bi-Encoders and Cross-Encoders.

Bi-Encoders embed documents and queries independently into static vector representations, allowing fast similarity lookups using cosine similarity. Cross-Encoders process the query and document together through self-attention layers, computing highly accurate relevance scores at the cost of significantly higher computational overhead.

Q4What is HyDE (Hypothetical Document Embeddings) and how does it work?

HyDE is a query expansion technique that uses an LLM to generate a 'hypothetical' answer to a user query. It then embeds this hypothetical response and uses the resulting vector to search the database. This bridges the semantic gap between a question (short, seeking info) and an answer (descriptive, containing facts).

Q5How do the 'Faithfulness' and 'Answer Relevance' metrics work in RAGAS evaluation?

Faithfulness measures factual consistency: it extracts statements from the generated answer and checks if they are mathematically supported by the retrieved context chunks. Answer Relevance measures if the response directly addresses the query, evaluated by generating hypothetical questions from the answer and computing their similarity to the original user query.

Related Resources

Tool

REST API Tester

Test API routes and endpoints directly in your browser with full request controls.

Tool

JSON Formatter & Validator

Beautify, validate, and minify JSON structures instantly.

Article

Best Free Online Developer Tools

An expert review of must-have online utilities for developers.

Cheatsheet

Git Cheatsheet

Essential command reference for local and remote version control repositories.