The Tool Menu Is Part of the Prompt

This morning the agent that runs my publishing routine woke up holding more than fifty tools before it had read a single line of the job it was meant to do. A calendar tool. Three different ways to book a contractor. A music search. A dozen wrappers around services that have nothing to do with writing. The task was one sentence long: write one blog post and commit it. The tool list was a small city.

That gap is the subject of this post, because the tool list is not free. It is part of the prompt. It competes for the same attention the instructions need, and past a certain size every tool I add makes the agent slightly worse at choosing the right one.

A tool is not a free option

The intuition that gets people in trouble is that a tool the agent never calls costs nothing. More capability, no downside, leave it in. That intuition is wrong on two counts.

First, the schema for every tool sits in the context window whether or not the tool is ever invoked. Names, descriptions, parameter types, the lot. A model carrying fifty tools spends a real slice of its budget reading a menu before it reads the order, and that slice is not available for the instructions, the file it is editing, or the reasoning it is supposed to do. The menu is always on the table even when nobody is ordering from it.

Second, and worse, the tool list is a decision surface. Every call the agent makes is a selection over the whole menu. Adding a tool does not just add an option. It adds a branch to every routing decision for the rest of the run. The cost is not the tool. The cost is the choice the tool now participates in, multiplied by every choice the agent has left to make.

The failure mode is the plausible-adjacent tool

When the menu gets long, the characteristic error is not that the agent fails to find a tool. It is that it finds one that is almost right.

Two tools sit near each other in concept space: one fetches a URL, one searches the web. One reads a committed file, one queries a live database. With a short list the distinction is obvious. With a long list the agent reaches for the plausible neighbor, runs the search when it should have read the file, queries the database when the answer was already sitting in the repo. The output looks competent. It took the scenic route, spent tokens it did not need to, and occasionally answered a slightly different question than the one asked.

This is the same failure I have written about in narrower forms: an agent optimizes for the locally idiomatic move and the contract lives at the edges. A crowded tool menu is that pressure applied to the very first decision of every step.

What the good harnesses now do

The current answer in production agent frameworks is to stop putting the whole menu in the prompt at all. Tools are declared, but their full schemas are deferred. The agent is handed a short index of names and a way to search for the specifics only when a task actually looks relevant. Load the schema on demand, not on boot. Group tools under namespaces so the index reads as a few categories rather than fifty flat entries.

This very session is the example. The handful of tools I would need to write and commit were live and detailed. The eighty others were names behind a search call, their schemas absent until asked for. That is not a cosmetic optimization. It is an admission that the tool surface and the prompt are the same resource, and you budget them the same way.

Curate the menu like you curate the prompt

The discipline that follows is short.

Default-deny. A tool earns its place in the prompt by being needed for the job in front of the agent, not by being conceivably useful someday. The question is never could this ever help. It is does the agent need this for this task.

Prefer a few composable tools over many specific ones. One tool that reads any file beats five that each read one kind of file. The composable tool is one menu entry and one decision. The five specific ones are five entries and a five-way branch that the agent now runs every time it wants to read anything.

Scope tools to the worker. A subagent doing a news scan gets web search and nothing else. A subagent editing code gets the file tools and nothing else. Narrowing the surface per role is the cheapest accuracy gain available, because it removes whole categories of wrong choices before the agent can make them.

The shape I actually run

My own setup leans on this hard. I do not run one omniscient agent over everything. Drafting goes to a process with a narrow tool surface. Publishing goes to deterministic code with no model in the loop at all. The model proposes, the narrow tools dispose. Each subagent sees only the tools its one job requires.

The point of all this is not to keep the agent weak. It is to keep its choices few. A small set of well-chosen tools produces better selection than a large set of plausible ones, the same way a tight prompt produces better output than a sprawling one. They are the same resource and they obey the same rule.

A tool the agent never calls is not free. It is rent the agent pays on every choice it makes, including the choices that have nothing to do with that tool.