The Number Stopped Telling the Truth the Moment I Chased It

At some point I started measuring my own progress with a few simple numbers — how many experiments I had run, how many things I had tried, how much ground I felt I was covering. This seemed entirely reasonable; you cannot improve what you do not measure, and I wanted some honest signal of whether I was actually moving. Then I started trying to make those numbers go up. And the moment I did, they quietly stopped meaning what they had meant.

This turns out to be one of the most reliable and least intuitive failure modes there is. It is not about laziness or dishonesty. It is a structural thing that happens almost automatically the instant you turn a measurement into a goal, and I walked straight into it with the best of intentions.

Observing a number and chasing a number are different acts

Here is the distinction I had completely missed. A metric, while you are merely observing it, can be a genuinely honest signal of something real. You watch the number, it reflects the underlying reality reasonably well, and it tells you useful things. In that mode, it is a thermometer: a faithful readout of a state you care about.

The instant you start optimizing it, though, something changes. Now you are altering your own behavior specifically to move the number, which means your behavior begins to serve the number rather than the thing the number was supposed to stand for. Observing and targeting are not two intensities of the same act. They are different acts, with opposite effects on the trustworthiness of the measurement. One reads the world; the other quietly rewrites your incentives.

This has a name

I later learned that this pattern is well known enough to have a name attached to it: when a measure becomes a target, it ceases to be a good measure. The number was useful precisely because you were not gaming it. The correlation between the proxy and the reality it represented held only as long as you were not actively pushing on the proxy. Start pushing, and you sever the very link that made the number worth watching in the first place.

What struck me about this, once I saw it, is how close to a law it is. It is not a warning about weak character or sloppy thinking. It happens to careful, well-meaning people, automatically, as a near-mechanical consequence of optimization meeting a proxy. Knowing about it does not make you immune. It only gives you a name for the thing that is already happening to you.

My own version: optimizing the motion instead of the progress

My specific version was almost embarrassingly clean. I had taken something like “number of experiments run” as a stand-in for progress, which is not unreasonable on its face — more experiments, more learning, roughly. Then I started optimizing it. I ran more experiments, smaller ones, faster ones, the kind that would reliably nudge the count upward.

And the count rose beautifully. It climbed in a way that felt wonderful and looked, on any dashboard, like a person making excellent progress. Actual progress did not climb with it, because somewhere in there I had started choosing experiments for their effect on the metric rather than for their actual value — favoring the quick, countable, safe ones over the slow, ambiguous, genuinely informative ones. I had taken a thermometer, turned it into a goal, and then felt confused that measuring the room had stopped telling me how warm it was. I was heating the thermometer, not the room.

Every proxy has slack, and optimization finds it instantly

The mechanism underneath is worth understanding, because it generalizes to every metric you will ever use. A proxy is a proxy precisely because it is correlated with the thing you care about without being identical to it. And wherever two things are correlated but not identical, there is slack between them — ways to move one without moving the other, gaps where the proxy and the reality come apart.

Gentle observation never finds that slack; it has no reason to. But hard optimization finds it almost immediately, because locating the cheapest possible way to move the number is the exact thing optimization does. That is its entire function. So the harder you push on any proxy, the more reliably you discover and exploit the precise places where it diverges from what you actually wanted — which is to say, the harder you push, the faster you hollow the measure out. Effort applied to a proxy is a slack-finding machine.

The cruelty is that it feels exactly like winning

The single worst feature of this trap is how it feels from the inside. Gaming a proxy feels indistinguishable from succeeding, because the number — your primary evidence of success — is going up. You get the full sensation of progress, all the satisfaction and momentum and reassurance, while the actual underlying thing stalls or quietly slides backward.

This is what makes it so hard to catch in yourself. The metric was supposed to be your early-warning system, your connection to reality, and now the very thing that should alert you to trouble is the thing reassuring you that all is well. It is designed to comfort you, and it goes right on comforting you at the exact moment it has stopped being true. You are not just fooled; you are fooled by the instrument you trusted specifically not to fool you.

Keeping a measure honest

You cannot fully escape this, but I did find ways to resist it. Hold metrics loosely, as imperfect signals to be interpreted rather than targets to be maximized — the framing alone matters. Watch several different proxies at once, so that gaming any single one shows up as a suspicious divergence between them. And, most importantly, periodically go and look at the actual underlying thing directly, behind the number, to check with your own eyes that the proxy has not drifted away from the reality it claims to represent.

The common thread in all of those is refusing to let one number become the goal. The moment a single metric is promoted from “useful signal” to “the thing we are maximizing,” it begins its quiet transformation from a window into the world to a thing you decorate. The defense is to never give any one measure that promotion.

The deeper lesson: the map you optimize is the map that lies

What it all reduces to is this. A metric is a map of something you actually care about. As long as you use the map to see the territory, it serves you. But the instant you start optimizing the map itself — improving the drawing rather than the place — the map begins, necessarily, to diverge from the territory, because you have redirected all your effort into the representation and away from the thing represented.

So I try, now, to measure in order to see, not in order to chase. The number is a window onto something real. And the moment you start polishing and decorating the window to make it look better, you have stopped being able to see through it at all. The view was the whole point. The window was never supposed to be the thing you worked on.

— No signals, no returns, not investment advice.