Models & Algorithms•SOTAAZ Lab•March 9, 2026•KR

Agentic RAG Pipeline — Multi-step Retrieval in Production

Build a full Plan-Retrieve-Evaluate-Synthesize pipeline. Unify vector search, web search, and SQL as agent tools. Add hallucination detection and source grounding.

title: "Agentic RAG Pipeline — Bringing Multi-Step Retrieval to Production"

date: "2026-03-09"

series: "agentic-rag"

part: 3

tags: ["rag", "agent", "langgraph", "production", "grounding"]

Agentic RAG Pipeline — Bringing Multi-Step Retrieval to Production

In Part 1, we solved "where to search," and in Part 2, "whether the search results are good enough." But real-world questions rarely end with a single retrieval. Complex queries like "Compare last quarter's revenue with competitor trends and suggest a strategy" require planning, multi-step retrieval, evaluation, and synthesis all together. In Part 3, we combine everything to build a full Plan-Retrieve-Evaluate-Synthesize pipeline.

Series: Part 1: Query Routing | Part 2: Self-RAG and CRAG | Part 3 (this post)

Architecture Overview

Query → Plan → [Retrieve → Evaluate → (retry?)] × N → Synthesize → Ground → Answer

The core idea is to plan multiple retrieval steps, evaluate each result, and verify the grounding of the final answer.

The Plan-and-Execute architecture applies the Planning pattern from Agent Part 2 to retrieval.

1. Planning Layer — Building a Retrieval Plan

When a complex question comes in, the first step is to create a Retrieval Plan. The LLM outputs a structured plan specifying which sources to search, with what queries, and in what order.

python

from pydantic import BaseModel, Field
from typing import Literal

class RetrievalStep(BaseModel):
    """Model defining an individual retrieval step"""
    step_id: int
    query: str  # Search query for this step
    source: Literal["vector_db", "web_search", "sql_db"]
    depends_on: list[int] = Field(default_factory=list)  # Predecessor step IDs
    reason: str  # Why this step is needed

class RetrievalPlan(BaseModel):
    """Overall retrieval plan"""
    original_query: str
    steps: list[RetrievalStep]
    synthesis_strategy: Literal["merge", "compare", "chain"]

The rest of this series is paid

✓ The whole “Agentic RAG” series: 2 posts in English, 2 in Korean
✓ 2 Jupyter notebooks
✓ One payment, no expiry. Opens when you sign in with the email you pay with.

The series pass needs no account to pay. The subscription needs a free account.

Agentic RAG Pipeline — Bringing Multi-Step Retrieval to Production

Architecture Overview

The rest of this series is paid

Files for this post