When a RAG system answers badly, teams reach for a bigger model. Nine times out of ten the model faithfully summarized the wrong chunks, because retrieval quietly failed three stages earlier. Most RAG failures are retrieval failures wearing a trench coat.
The live demo on this site runs a complete production-shaped pipeline, small but real. This is a walk through it, stage by stage, with the number that matters at each step and where each number comes from.
Ingest: assume hostile documents
Documents arrive broken. Scans without text layers, PDFs with mangled encodings, files that are secretly zip bombs of tables. The parse and OCR stage exists to turn that mess into text, and it will fail on some inputs no matter how good it gets.
The design decision is not "parse everything." It is "fail loudly per document, never per pipeline." One poison document must not take down ingestion for everyone behind it in the queue.
Chunk: the cap is a safety device
Chunking splits documents into retrievable pieces. Every tutorial covers chunk size. Almost none cover the failure mode that matters: the pathological document that explodes into tens of thousands of chunks and eats your embedding budget on one file.
The demo caps chunks per document. Hitting the cap raises a terminal error and routes the document to a dead-letter queue, where a human can look at it. That is the same lesson as the incident log: the system does not get to decide that an anomaly is fine. It parks the anomaly and tells someone.
Embed: dimensions are a cost dial
Each chunk becomes a vector. The demo runs gemini-embedding-001 at 768 dimensions. A recruiting platform we operate runs text-embedding-3-large at 3,072 dimensions. Neither choice is "better." They are different points on a cost-quality curve, and the honest way to pick is to run both against your own corpus and measure retrieval, which is exactly how the demo's embedding migration was validated before it shipped.
Two production notes the tutorials skip. First, dimensions multiply through everything: storage, index size, query latency, and the embedding bill itself. At corpus scale, 768 versus 3,072 is real money. Second, persist the token count per embedding. On a studio-scale ETL we run, that per-embedding accounting is what makes per-tenant cost attribution possible. Without it, your AI bill is one opaque number nobody can act on.
Index: pgvector until it isn't
The demo indexes into Postgres with pgvector. A due-diligence platform we built does the same over PE/VC questionnaire corpora. When the workload demanded scale and richer keyword machinery, a recruiting platform went to AWS OpenSearch instead, where hybrid queries hold p95 under 150 ms on the candidate corpus.
The pattern that has aged best: make the index a swappable backend behind one interface. The demo ships two implementations behind the same contract, which turned "should we change the index?" from a rewrite into a config change with an eval run.
Retrieve: hybrid, fused with RRF
Pure vector search has a blind spot that shows up on day one with real users: exact strings. Part numbers, person names, statute references, error codes. Embeddings are built to blur meaning together, which is precisely wrong for ERR_MODULE_NOT_FOUND.
So production retrieval runs two searches, semantic and keyword, and fuses the rankings with Reciprocal Rank Fusion at k=60. RRF's virtue is that it needs no tuned weights: a document ranked well by either method surfaces, and k=60 stops any single list from dominating. On the demo's eval set, that hybrid stack holds recall@k at 0.93. The due-diligence platform holds recall@5 at or above 0.92 on real questionnaire corpora.
Recall is the number to obsess over, because it is the ceiling on everything downstream. The model cannot cite a chunk retrieval never fetched.
Generate: the model is the last stage
Only now does an LLM enter. It gets the question, the retrieved chunks, and instructions to answer from them with citations. On the demo, this stage costs about 9 seconds, most of it the reasoning model thinking. That is the loudest, most visible stage of the pipeline, and it is the one that was working fine all along in most "our RAG is bad" postmortems I have been called into.
The gate: a number without CI is a screenshot
Here is the part that separates production RAG from a notebook. The 0.93 is not a number from launch week. The repo carries an eval suite with recorded judge cassettes, and it runs on every pull request. The CI comment shows the recall delta against main. If a change drops retrieval quality, the merge is blocked before a customer ever sees the regression.
That is what "measured, not assumed" means mechanically. Chunking changes, embedding migrations, index swaps, retrieval tweaks: all of them face the same gate. The number stops being a claim and becomes a property of the system, enforced the same way type checks are.
The first article in this series argued that the winning method is a narrow loop with an eval in front and a human behind. This post is that sentence unpacked into plumbing. The orchestration layer underneath it has its own article, the full runbooks and ADRs live in the operator brief, and the pipeline itself is one click away: drop in a document and watch every stage of this post run against it.