Agentic RAG Pipeline — Multi-step Retrieval in Production
Build a full Plan-Retrieve-Evaluate-Synthesize pipeline. Unify vector search, web search, and SQL as agent tools. Add hallucination detection and source grounding.
title: "Agentic RAG Pipeline — Bringing Multi-Step Retrieval to Production"
date: "2026-03-09"
series: "agentic-rag"
part: 3
tags: ["rag", "agent", "langgraph", "production", "grounding"]
Agentic RAG Pipeline — Bringing Multi-Step Retrieval to Production
In Part 1, we solved "where to search," and in Part 2, "whether the search results are good enough." But real-world questions rarely end with a single retrieval. Complex queries like "Compare last quarter's revenue with competitor trends and suggest a strategy" require planning, multi-step retrieval, evaluation, and synthesis all together. In Part 3, we combine everything to build a full Plan-Retrieve-Evaluate-Synthesize pipeline.
Series: Part 1: Query Routing | Part 2: Self-RAG and CRAG | Part 3 (this post)
Architecture Overview
Query → Plan → [Retrieve → Evaluate → (retry?)] × N → Synthesize → Ground → AnswerRelated Posts
TurboQuant in vLLM on One A100 — Capacity, Speed, and Accuracy of All Four Presets on an 8B Model
vLLM 0.28, Qwen3-8B bf16, one A100 80GB: KV capacity, batched throughput, 32K decode, needle-in-haystack, and GSM8K for bf16, fp8, and all four TurboQuant presets — the 8B size vLLM's own study skipped.
TurboQuant From Scratch on Real KV Tensors — What 3 Bits Actually Cost, and Why the Forks Beat the Paper's Layout
PolarQuant in 60 lines of PyTorch on real KV from Llama-3.2-1B and Qwen3-8B: 3-bit costs +10% perplexity, k8v4 +0.2%, QJL only pays below 4 bits, and the block-32 layout explains half the forks' edge.
TurboQuant llama.cpp CUDA Fork, Measured on an A100 — turbo4 Matches q4_0, turbo3 Breaks at Long Context
Qwen3-8B Q4_K_M on one A100, six KV types: perplexity, prefill, decode-at-depth, and VRAM measured. turbo4 matches q4_0 quality and beats q8_0 decode 2.5x at depth; turbo3 triples perplexity at 32K context.