--- title: 'Inside an MCP tool server: what 850 tools teach you' excerpt: 'MCP server architecture, explained from production: at 850 tools the real work is selection, auth scoping, caching, and loading skills before tools.' date: '2026-06-13' lastModified: '2026-07-04' author: 'Teo Deleanu' authorAvatar: '/team/teo.jpg' tags: ['Production AI', 'AI Engineering', 'MCP', 'Agents'] keywords: ['MCP tool server architecture', 'agent tool selection', 'MCP authentication scoping', 'tool catalog context cost'] featured: false tldr: 'An MCP tool server is an HTTP service that speaks a schema the model was trained to follow. On a financial-analytics platform we run 22 hubs with 850+ tools, and the protocol code is the smallest part of the system. The engineering that matters is tool selection (semantic reranking, catalog compression), scoped user/admin auth, Redis caching with per-tool TTLs, and loading versioned skills before tools. Each exists because agents consume APIs differently than human developers do.' keyTakeaways: - 'At 850 tools the hard problem is selection: semantic reranking and a compressed catalog beat dumping raw definitions into context every turn' - 'Tools execute as someone. Scoped user and admin servers with JWT are a stronger guarantee than asking the model to behave' - 'Agents repeat lookups, so cache at the tool layer: Redis L2, SHA256 keys, per-tool TTLs, p95 swap quotes under 300 ms' - 'Load versioned skills before tools so the model knows how to use what it is about to call' --- An MCP tool server is an HTTP service that speaks a schema the model was trained to follow. That is the entire trick. A tool is a name, a JSON Schema for its arguments, and a handler function; the transport is JSON-RPC. If you have shipped a REST API, you can ship a working MCP server in a day. We run this pattern at two different scales. A financial-analytics platform we work on exposes 22 MCP hubs with more than 850 tools for token resolution, asset search, swap quotes, and oracle and gas data. An IoT platform we built runs one TypeScript server on the official MCP SDK, 42 tools across 5 domains. Between the two, almost none of the hard engineering sits in the protocol layer. It sits in selection, auth scoping, caching, and what the model reads before it calls anything. ![Anatomy of an MCP tool server: agent to semantic tool selection to scoped auth to execution to Redis cache, with skills loading first](/blog/mcp-server-anatomy.svg) ## The protocol layer is a weekend project The IoT server is the clean baseline. TypeScript on the official MCP SDK, 42 tools split across admin, energy, environmental, health, and weather domains. Each tool is a schema plus a handler that queries the platform's own API. OpenTelemetry auto-instrumentation and structured logs give us a trace per tool call without custom plumbing. Nothing in that list is exotic. If the protocol part of MCP feels hard, you are probably overbuilding it. The lessons start when the catalog grows, because agents consume APIs differently than human developers do. A developer reads your docs once and calls three endpoints. An agent re-reads the full tool catalog on every turn and might call anything in it. ## At 850 tools, the hard problem moves from execution to selection The fintech platform's five largest hubs hold 188, 117, 103, 67, and 34 tools. No model reliably picks the right one from 850 raw definitions, and the failure has two parts. Accuracy first: the catalog contains near-neighbors, and a model choosing between an asset-search tool and a token-resolution tool on names and descriptions alone will guess. Cost second: tool definitions ride into the context window on every single turn, so a fat catalog is a tax you pay per turn, forever. The platform handles selection with semantic reranking over the catalog. Embed the task, shortlist candidate tools, and hand the model a page instead of a phone book. A fast mode trades reranking depth for latency when the shortlist is obvious. A different internal platform we run pushed the diet further and replaced 200+ hardcoded tool definitions with one generic tool backed by a compressed 9k-token catalog the agent searches at runtime. Before, every turn paid for 200 definitions whether the agent needed them or not. The per-token math is the same math that decides whether self-hosting a model pays off, which I walked through in [the cost meter breakdown](/blog/free-because-open-source). Catalog size is a recurring cost, and you should meter it like one. ## Tools execute as someone "The model asked for it" is not an authorization model. On the IoT platform every tool call carries a JWT, and multi-tenant scoping happens at the server level: regular users connect to a user-scoped MCP server, admins to an admin-scoped one. The user server does not contain the admin tools at all. A prompt injection cannot invoke a tool that was never in the catalog, which is a much stronger guarantee than a system prompt asking the model to behave. The trade-off is real. There are two server variants to build and keep in sync, and every new tool forces a permissions decision at design time. We accept that, because reviewing a tool's scope once up front is cheaper than auditing what an agent did with an over-scoped credential at 03:00. ## Cache at the tool layer, because agents repeat themselves Agents hammer the same lookups. One analysis session on the fintech platform can resolve the same token and re-quote the same pair dozens of times, and parallel agents multiply that. The platform runs a Redis L2 cache in front of tool execution. Keys are SHA256 hashes of the normalized request, so identical calls from different agents and sessions land on the same entry. TTLs are set per tool: token metadata can live for hours, a gas price only for seconds. The numbers hold up under that design: p95 swap-quote latency stays under 300 ms and the API runs at 99.9% availability, with the cache absorbing repeat traffic that would otherwise hit upstream oracles and aggregators. Caching inside the agent loop would miss cross-session hits. The tool layer is where the requests converge, so that is where the cache belongs. ## Load the expertise before the tools A tool definition tells the model what it can call. It says nothing about how to use the answer. The IoT platform loads skills before data: when an agent starts an energy or environmental task, the server loads the domain framework (baselines, thresholds, which metrics matter in which season) before any query tool runs, so the model interprets readings instead of echoing them. Those skills are versioned artifacts rather than prompt folklore. We maintain roughly 85 SKILL.md files under an ADOPT/FORK/BUILD governance rule, semver-pinned so a stale dependency fails in CI instead of surfacing as a confidently wrong answer in production. Customer-facing skills stay segregated from team-internal ones. It is the same reflex as [treating evals as CI](/blog/evals-as-ci): encode the judgment in a versioned file and let the pipeline tell you when it rots. ## What the contract buys you MCP standardized the doorway, and that is worth a lot: one schema that any client and any trained model can speak. Everything in this post lives behind that doorway, and none of it is in the spec. Selection, scoping, caching, and skills are your problems at any catalog size; they simply become unavoidable somewhere past the point where a human can still read the whole tool list. [The RAG deep-dive](/blog/how-rag-works-in-production) covers what happens after the tools return data. The systems above are described in more detail in [the operator brief](/about), and the [live demo](/demo) is the public, single-service end of the same practice. If you are building a tool server past a few dozen tools, start with the reranker and the auth split. The protocol will take you a day. The catalog will take you a quarter.