LlamaIndex

Overview

LlamaIndex is a data framework for building RAG pipelines, knowledge assistants, and data-augmented LLM applications. It provides document loading from 300+ sources, flexible chunking strategies, multiple index types, hybrid retrieval with reranking, and production evaluation tools for question-answering systems.

Instructions

• When ingesting documents, use

  SimpleDirectoryReader

  for local files or LlamaHub connectors for SaaS platforms, and run through an

  IngestionPipeline

  with metadata extractors (title, summary) and deduplication.
• When chunking, start with

  SentenceSplitter

  at 1024 tokens with 200 token overlap, use

  MarkdownNodeParser

  for structured documents,

  CodeSplitter

  for code, and adjust based on evaluation results.
• When indexing, use

  VectorStoreIndex

  as the default for most RAG,

  KnowledgeGraphIndex

  for entity relationships, and

  DocumentSummaryIndex

  for per-document summaries.
• When retrieving, implement hybrid retrieval (vector + keyword) for production, add a reranker (

  CohereRerank

  ) after retrieval for improved relevance, and set

  similarity_top_k

  based on context window (3-5 for large models, 2-3 for smaller).
• When building query engines, use

  RetrieverQueryEngine

  for standard RAG,

  CitationQueryEngine

  for responses with source attribution, and

  SubQuestionQueryEngine

  for complex multi-part queries.
• When creating agents, use

  ReActAgent

  with tools wrapping query engines (

  QueryEngineTool

  ), functions, and other agents for multi-step reasoning.
• When evaluating, use

  CorrectnessEvaluator

  ,

  FaithfulnessEvaluator

  , and

  RelevancyEvaluator

  on a test set before deploying.

Examples

Example 1: Build a RAG pipeline over company documentation

User request: "Create a question-answering system over our internal docs"

Actions:

1. Load documents with

   SimpleDirectoryReader

   and extract metadata (title, summary)
2. Chunk with

   SentenceSplitter

   (1024 tokens, 200 overlap) through an

   IngestionPipeline

3. Create

   VectorStoreIndex

   with OpenAI embeddings and configure hybrid retrieval
4. Build

   CitationQueryEngine

   for answers with source references

Output: A RAG system that answers questions with citations from company documentation.

Example 2: Create a multi-source research agent

User request: "Build an agent that can search across our docs, database, and web"

Actions:

1. Create separate query engines for each data source (vector index, SQL, web search)
2. Wrap each engine as a

   QueryEngineTool

   with descriptive tool descriptions
3. Build a

   ReActAgent

   that routes questions to the appropriate tool
4. Add

   SubQuestionQueryEngine

   for complex queries requiring multiple sources

Output: An intelligent agent that reasons about which data source to query and synthesizes multi-source answers.

Guidelines

• Use

  SentenceSplitter

  with 1024 token chunks and 200 token overlap as the starting point.
• Always add metadata extractors to the ingestion pipeline; title and summary metadata improve retrieval significantly.
• Use hybrid retrieval (vector + keyword) for production; pure vector search misses exact term matches.
• Add a reranker (

  CohereRerank

  ) after retrieval to improve result relevance for small cost.
• Evaluate with

  CorrectnessEvaluator

  on a test set before deploying; subjective quality assessment does not scale.
• Set

  similarity_top_k

  based on context window: 3-5 chunks for large models, 2-3 for smaller models.
• Use

  IngestionPipeline

  with deduplication for incremental data updates; do not re-embed unchanged documents.