Reading a 500,000-token corpus into the prompt costs $0.15 per query at the $0.30 per million input rate the demo's model publishes. Embedding that same corpus once with gemini-embedding-001 costs about $0.08 at $0.15 per million tokens, and the spend never recurs. Same corpus, same provider, and at 1,000 queries a month one approach bills $150 while the other bills pennies plus the cost of keeping an index honest. That arithmetic, done in advance, is most of what this post has to say.
ByteByteGo's EP220 gave a solid conceptual tour of RAG, agentic RAG, and long-context models. This is the version with production numbers, from systems we run. The claim: these are three points on one cost and quality curve, and what distinguishes them is where the meter sits. Big context bills per query. RAG moves the spend into an index that amortizes across queries, and agentic retrieval bills per model call, per question. Each wins somewhere specific. Picking by hype instead of by corpus size and query shape is how retrieval budgets die.
Big context: the meter runs on every query
The mechanism could fit in a tweet. Put the documents in the prompt, put the question after them, and let the model read everything. No parser, no chunker, no vector database, no retrieval stage to debug. When a document changes, the next query sees the new version, because there is nothing to re-index. Every retrieval failure mode I catalogued in how RAG works in production simply does not exist here. The model cannot miss a chunk it was handed directly.
You pay for that simplicity on a linear meter. Input cost is corpus size times the per-million rate, charged on every query, forever. The 500,000-token corpus above costs $0.15 per question before the model writes a single output token; at 10,000 questions a month that is $1,500 for re-reading the same bytes. Latency rides the same curve, since the model ingests the full prompt before the first output token appears. And there is a quality caveat that deserves careful wording: long-context models are known to recall content unevenly across the window, with material buried mid-context faring worst. Providers acknowledge the behavior and it varies by model. I am not attaching a number to it because we have not measured it on our own corpora, and this blog does not quote benchmarks we did not run.
So big context wins in a specific spot: a corpus that is small, stable, and queried rarely. A single product manual asked twenty questions a month does not deserve a pipeline.
Classic RAG: the meter moves into the index
The live demo on this site is the receipt for this lane. The corpus is parsed, chunked, and embedded with gemini-embedding-001 at 768 dimensions, at $0.15 per million tokens, into pgvector. A query then costs a question-sized embedding and a database lookup: two searches, semantic and keyword, fused with Reciprocal Rank Fusion at k=60. On the demo's eval set that hybrid stack holds recall@k at 0.93, and the number is enforced rather than remembered, because the eval suite runs on every pull request and a recall drop blocks the merge. The stage-by-stage walk through that pipeline is its own article; I will not repeat it here.
What matters for this comparison is the shape of the bill. The index has real carrying costs: you embed the corpus up front, you re-embed documents when they change, and you maintain the eval gate that proves retrieval still works after every chunking tweak or embedding migration. But all of that is paid per document or per change, while the per-query meter stays tiny and flat regardless of corpus size. A million-document corpus and a hundred-document corpus cost roughly the same per question. The index is a capital expense that amortizes, and amortization needs volume. High query counts over a large or changing corpus is exactly where the math turns in RAG's favor, and exactly where the big-context meter becomes indefensible.
Agentic retrieval: the meter runs per model call
First the honesty clause. We do not run any system marketed as "Agentic RAG". The honest receipt is the Analytical agent on an IoT analytics platform we built, which runs on Claude Opus 4.8 and turns natural-language questions into SQL over telemetry. Mechanically, that is what agentic retrieval means: the model decides what to fetch, inspects what came back, and can fetch again. Nobody on that platform uses the buzzword. The behavior is the same thing the buzzword describes.
This lane exists because some questions have no chunk to retrieve. "How many devices reported errors last week" is a computation over rows, and no pre-embedded passage contains the answer. Retrieval has to be constructed at question time, by something that understands the schema and can write the query. That is a fetch decision delegated to a model, which is precisely what the first two strategies avoid paying for.
The meter follows the delegation. Every question pays a routing decision on Claude Sonnet 4.6 plus at least one specialist call on Opus 4.8, and a question that needs iteration pays for each round trip. Two design choices keep that bill sane on the platform. Single-domain questions, the common case, bypass synthesis entirely and cost one routing decision plus one specialist call, so only the multi-domain minority pays for the full machinery. And safety is enforced in code rather than prompts: every generated query is parsed with sqlglot and the tenant's application_id predicate is injected into the AST before execution, because a model that composes its own queries needs a boundary it cannot vote on.
Pick by corpus size and query shape
The framework falls out of the meters.
Small stable corpus, low query volume: big context. If corpus tokens times the input rate times monthly queries stays below what an index costs to build and keep honest, the pipeline is overhead. The demo would have shipped this way if it served one manual to twenty queries a month.
Large or changing corpus with real query volume: RAG. The index amortizes, the per-query cost stays flat, and the eval gate is what makes the recall number a property of the system rather than a launch-week screenshot. This is the demo's actual shape.
Questions that need multi-step lookups or live structured data: agentic retrieval. You accept a per-question bill in model calls because no static index can answer an aggregation. Then you engineer the bill down the way the IoT platform does, with a mid-tier router in front and a bypass for the common case.
Corpus size and query shape are measurable before a line of code exists, which means this decision costs an afternoon of arithmetic. The systems behind all three lanes are summarized in the operator brief, and the RAG lane is running at the live demo if you want to drop a document in and watch the amortized meter tick.