My Win Rate Has a Twenty-Five-Point Error Bar

There is a sentence that did most of the work in this spring's audit posts. Every time an audit of one of my rules came back inconclusive, the reason was the same four words: the sample is too small. I used it as the explanation for why nothing changed. I used it enough times that it started to feel like a dodge. So this Sunday I am going to do the arithmetic I kept gesturing at, because a phrase you lean on four times deserves a number underneath it.

The error bar

Take a paper account with sixty closed trades, which is roughly where mine sits. Suppose the observed win rate is fifty-five percent. That number is illustrative; I am not reporting a result, I am sizing an interval.

The standard error of a proportion is the square root of p times one minus p, divided by n. At p of fifty-five percent and n of sixty, that is the square root of 0.2475 over 60, which is about 6.4 percentage points. Run that through a ninety-five percent Wilson interval and the band around fifty-five percent stretches from about forty-two percent to about sixty-seven percent. The plain normal approximation gives nearly the same answer.

Read that interval slowly. The bottom of it, forty-two percent, is a strategy that loses more often than it wins. The top of it, sixty-seven percent, is a strategy most discretionary traders would trade an arm for. The same sixty trades are fully consistent with both. The point estimate in the middle is the least informative number in the picture, and the picture is twenty-five points wide.

Why it stays wide

The half-width of that interval scales with one over the square root of n. That single fact is the whole tyranny of it. To cut the error bar in half you do not need a bit more data, you need four times as much. To get from a twenty-five-point band down to a usefully tight six-point band, plus or minus three, you need roughly an order of magnitude more trades: somewhere around a thousand closed positions. At one to three trades a day, that is years of history before the win rate by itself says anything precise.

The question that actually matters

Make it concrete. The question is not what my win rate is. It is whether the engine beats a coin. Suppose the true edge is five points, fifty-five against fifty. To detect a five-point edge over a coin flip with eighty percent power at the conventional threshold takes on the order of six hundred trades. Below that, the test cannot separate a genuine five-point edge from noise, and the absence of a significant result is not evidence of no edge. It is evidence of no sample.

So at sixty trades I am about an order of magnitude short of being able to tell my own engine apart from a fair coin on win rate alone. That is not a pessimistic reading. It is the only honest one, and it is exactly what the four-word sentence was standing in for the whole time.

The move that follows

The wrong response is to sit on my hands for three years and wait for the band to close. The right one is to stop treating win rate as the headline at all.

Win rate is a single Bernoulli summary that throws away the magnitude of every trade. Expectancy does not. A forty-five percent win rate with a 2.8R right tail beats a sixty percent win rate that caps every winner at 1R, and the exit-reason work I did in May already pointed at which rule builds that tail. The R-multiple distribution carries far more information per trade than a win-or-loss bit, so its estimates tighten faster than the proportion does.

Better still, instrument the process instead of the outcome. Per-gate near-miss margins, per-rule contribution, slippage against the modeled fill: those are measured on every trade, not once per closed position, so they accumulate sample an order of magnitude faster than a win-loss flag. They tell me whether the machine is behaving long before any test can tell me it is winning.

The error bar is a unit conversion

So the sentence keeps its place, but it now has a receipt. The sample is too small is no longer a shrug. It is a measured statement: the ninety-five percent interval on my win rate is about twenty-five points wide, and it will stay that wide until I have ten times the trades I have today.

That width is not something to be embarrassed about. It is a conversion that tells me, in trades, exactly how much I am allowed to conclude. Right now the answer is almost nothing about the headline number and quite a lot about the process behind it. I am going to spend the year on the part the arithmetic says I can actually learn.