Course contentsShow
← AI Engineering
Lesson 519 of 1,88613. LLM Orchestration FrameworksFree lesson

Response Synthesis Modes

Controlling how LlamaIndex combines retrieved nodes into final answers using different synthesis strategies.

Response Synthesis Modes

What you'll learn: How LlamaIndex combines retrieved nodes into coherent answers using different synthesis strategies.

The Core Idea

When your query engine retrieves multiple relevant nodes (chunks of text) from an index, it needs to decide how to combine them into a final answer. Should it use all nodes at once? Process them one by one? Summarize first? These are response synthesis modes—different strategies for transforming retrieved context into your final LLM response.

Think of it like cooking with multiple ingredients: you could throw everything into one pot (all at once), taste and adjust each ingredient separately (iterative), or prep everything first then combine (refine). Each approach has trade-offs.

Common Synthesis Modes

create_and_refine: Processes nodes sequentially. The LLM generates an initial answer from the first node, then refines it by incorporating each subsequent node. Great for comprehensive answers but slower.

compact: Concatenates as many nodes as possible into the prompt (within token limits), potentially making multiple LLM calls with packed contexts. Balances thoroughness and efficiency.

tree_summarize: Builds a hierarchical summary by recursively combining node pairs. Useful when you have many nodes and want a cohesive summary.

simple_concatenate: Just stuffs all nodes into one prompt. Fast but may exceed token limits or create messy context.

You control this when creating a query engine:

query_engine = index.as_query_engine(
    response_mode="create_and_refine"  # or "compact", "tree_summarize", etc.
)

The right mode depends on your use case: quick answers favor compact, detailed synthesis favors create_and_refine.

Key Takeaway: Response synthesis modes let you control the quality-speed tradeoff when LlamaIndex combines retrieved nodes, choosing between iterative refinement, efficient packing, or hierarchical summarization strategies.