Writing
Field notes on agentic AI, RAG, and shipping LLM systems that survive real customers. How the systems actually work, and how they fail.
The narrow loop works when the steps run in order. Seven of them, each anchored to real dates from the doc-intel demo: 29 days, 74 commits, $20 a month.
Live on June 27. Within four days the model was retired upstream, the UI failed silently, and the timeout budget was wrong. The week-one incident log.
Google retired our model snapshot three days after deploy. The fix was one config line, decided months earlier by three structural choices.
The scariest failure returned HTTP 200 on every request. Five silent-failure classes from production and the open-source stack that catches each one.
The agentic coding workflow behind 74 commits in 29 days: spec first, a fresh agent per task, TDD, two review passes, and an eval gate in CI.
Spec on May 29, live on Fly.io June 27, 74 commits, one engineer plus coding agents. The setup that makes review bandwidth the only constraint left.
The friction from idea to working software collapsed. The friction from working software to a system you can trust in production did not.
An ex-Meta L8 walked through his full agentic engineering workflow. My notes after 15+ years shipping: where we converge, and where my pager disagrees.
AI agents rarely fail because the model is weak. Six failure modes from systems we run, and every one of them lives in the layers around the model.
The generalist software lane is crowded and slow. From a hiring seat, demand for engineers who operate models in production is loud and supply is thin.
Most AI-engineer roadmaps list technologies. This one lists the six places production actually broke for us, because that is what the job defends against.
One production platform, end to end: agents, an MCP toolchain, a queue that survived 429 storms, and the numbers: 99.9% availability, p95 under 300ms.
Pavleur streams live meeting audio through Deepgram and ElevenLabs. Transcription turned out to be the easy part. The pipeline is mostly failure handling.
How to write LLM evals that run as CI: a golden set scored by recorded judge cassettes on every PR, deterministic and free, blocking merges on regressions.
Long context vs RAG vs agentic retrieval: one cost curve, three meters. The bill sits per query, in the index, or per model call. Corpus size picks.
MCP, RAG, and agents answer three different questions: knowledge in, actions out, control flow. A map of how production systems compose them, with receipts.
Context engineering for LLM-based agents, from production: 200+ tool definitions cut to a 9k-token catalog, with the reclaimed budget spent on skills.
A production-ready multi-agent architecture: a cheap router up front, expensive specialists behind it, safety in code, and a bypass for the common case.
The three types of AI agent memory in production: run-scale checkpoints, versioned skills, and source-hashed work memory. The context window is a cache.
MCP server architecture, explained from production: at 850 tools the real work is selection, auth scoping, caching, and loading skills before tools.
The RAG pipeline steps that decide quality, from a live system: parse, chunk, embed, index, retrieve, generate, with a measured number and CI gate at each.
The weights are free. Running them is not. Every AI system carries three meters: being ready, per call, and touching the world. Ours, with numbers.
We use cookies for analytics and ads measurement (incl. the Meta Pixel). Privacy policy.