The Cheapest Test Is the One You Run Before You Build

The most expensive engineering mistakes in my systems were not hard bugs. They were reasonable implementations of questions nobody had defined.

That distinction matters because the usual response to a bad result is to improve the implementation. Add retries. Increase context. Tune the threshold. Capture another metric. Sometimes that is the right move. Often it is a way to avoid asking whether the system was ever measuring the thing that mattered.

The cheapest test is the one you run before you build.

Start with the state transition

When I describe a feature, I now try to finish this sentence first: "After this succeeds, the observable state that changes is..."

For a trading workflow, the answer might be an order intent persisted with a unique identifier, followed by a broker-confirmed terminal state. For an agent workflow, it might be an allowlisted file changed and a postcondition checked against the resulting file. For a content pipeline, it might be a specific post committed to a specific repository.

If I cannot name the state transition, I do not yet have a feature. I have a desired feeling.

This test catches a surprising number of attractive designs. A dashboard that displays signals may be useful, but it has not demonstrated that a decision was made. A retry that returns HTTP 200 may have improved transport reliability, but it has not shown that the action is idempotent. A generated draft may be well written, but it has not been published.

The interface is not the output. The output is the verified change in state.

Separate the question from the mechanism

The next cheap test is to write down the question in a form that could be falsified.

"Make the agent smarter" is not a question. "Reduce refusals caused by missing context" is closer, but still needs an observable measure. "On a fixed evaluation set, preserve the same policy boundary while reducing tool-selection errors from the current baseline" is testable.

This sounds like process overhead until the first implementation fails. Then the wording becomes an escape hatch. If the original goal was vague, every partial improvement can be reported as progress. The system can be faster, more verbose, or more sophisticated without being more correct.

I have made this mistake in quantitative work too. A backtest can answer whether a set of recorded trades had a particular return. It cannot answer whether signals that never became trades were better opportunities. A report can be numerically precise and still be about the wrong population.

The practical fix is to define the population before looking at the result. Which events count? Which refusals count? What does a missing row mean? These are design decisions, not cleanup details.

Put the boundary in the test

A good test does not only check the happy path. It checks who or what is allowed to cause the change.

An agent may propose edits across a repository, but the routine may permit only one file to be staged. A market process may receive a valid signal, but the risk layer may refuse the order because exposure is already at its limit. A scheduled job may run, but a closed market window is a valid refusal rather than an error to retry.

The boundary should be explicit in both code and records. Otherwise a successful result proves only that the mechanism worked, not that it was permitted to run.

This is also where logs become more useful than screenshots. A screenshot shows what the system looked like after the fact. A durable decision record can show the input, the rule, the authority, the refusal or action, and the postcondition. That record makes the next test possible.

The uncomfortable career lesson

Engineers are rewarded for shipping visible things. A new route, a larger model, a faster query, and a polished dashboard are easy to point at. Defining the missing state transition is less visible, especially when it prevents a week of implementation from starting.

But the ability to stop early is not a lack of ambition. It is a form of technical judgment.

My current checklist is short:

1. What exact question is this system answering?
2. What state changes when it succeeds?
3. What state proves that it succeeded?
4. Which inputs can legitimately cause that change?
5. What is the valid refusal, and where is it recorded?

If those answers fit in a paragraph, implementation is usually straightforward. If they do not, adding code tends to make the ambiguity more expensive.

The best engineering improvement I made this week was not a new tool. It was moving the test to the front of the work. A ten-minute definition prevented an elegant solution to a problem I had not actually specified.