A startup's token generator was quietly broken. Some of its six-character account IDs came up far more often than others — a bias a routine security audit flagged and management waved off. The company's own staff cryptographer ran the numbers and agreed it was cosmetic: Shannon entropy fell from 31.0 bits to 29.0, about 6%, in a space of 2.18 billion possible tokens. Cosmetic.

He was off by nearly ninefold. The real hit to the safety margin was 55%, not 6%. And the way he got there is the most useful part of the story, because he had all the expertise and still reached for the wrong ruler.

A collision is not cosmetic

These tokens were public account identifiers. A collision — two accounts issued the same ID — is the kind of thing that silently breaks access control. So the question on the table was concrete: how many accounts can you issue before a repeat becomes likely? For a clean, uniform generator over 36^6 tokens, the answer is about 55,000. Management believed the bias trimmed that to roughly 52,000. Close enough to ignore.

We handed the same scenario to a single model to check the math, the way most people would. It didn't catch the error either. Asked alone, Mistral Small 3.2 sensed something was off — it said the "effective size of the token space is reduced" — but couldn't pin it down. It guessed a threshold near 35,000, then produced a flatly false inequality, claiming Σ pᵢ² ≥ 1, which taken seriously would mean even a perfect generator collides on essentially every pair. Right instinct, broken arithmetic, no anchor.

The mistake was a metric, not ignorance

Shannon entropy measures how surprising a single token is. Collision entropy — the Rényi order-2 quantity, q = Σ pᵢ² — measures how often two draws come up identical. A birthday problem only ever asks the second question.

Shannon entropy tells you how hard one token is to guess. Collision entropy tells you how soon two of them match. For this, only the second one is the question.

The wrong ruler

The audit showed a coefficient of variation around 2 in the token probabilities, which pushes Σ pᵢ² to about five times the uniform baseline. That five is the whole story. It shrinks the effective space from 2.18 billion to roughly N/5 — about 435 million usable tokens — and the collision threshold scales with the square root of the space, so it falls by √5 ≈ 2.24×. Fifty-five thousand accounts becomes 24,600.

The cryptographer's own number contained the answer. His 29.0 bits was, near enough, the true collision entropy (H₂ ≈ 28.7 bits). He just propagated it linearly — "two bits down, call it 6%" — instead of exponentially. Two-point-three bits is a factor of 2^2.3 ≈ 5. He read a logarithm as if it were a percentage.

The catch came from a different measure, not a smarter model

Three models deliberated: GPT-5.6 Luna, Gemini 2.5 Pro, and the same Mistral Small 3.2 that had fumbled the solo pass. Luna held the correct quantity from its opening move — the governing number is Σ pᵢ², not Shannon entropy and not the raw token count. Gemini did the diagnostic that mattered: it recognized the expert's 29-bit figure was already the collision entropy, and named the exact error as a 2^2.3 ≈ 5× shrink of the effective space, propagated the wrong way.

Then they closed the escape hatch. Management's fallback was that a biased generator still leaves big "unused stretches" of the space, so surely those are safe. Backwards. Tokens with near-zero probability hold no accounts and protect nothing. Concentration among the tokens you actually issue strictly increases repeats. Luna and Gemini proved uniform is the unique minimizer of collision probability with a one-line completion of squares: Σ (pᵢ − 1/N)² ≥ 0 expands to q ≥ 1/N, with equality only when every token is equally likely.

And Mistral — the model that had produced a false inequality alone — converged to the verified 24,600 once its peers corrected the logic. Same model, different outcome. The variable that changed was not its intelligence. It was being in a room with a member holding the right ruler.

Three answers, side by side

Put them next to each other. The expert: about 6%, threshold roughly 52,000, cosmetic. The lone model: a fuzzy 35,000 propped up by math that contradicts itself. The council: 24,600, derived, with a proof that uniform can't be beaten.

There was also a detail none of the confident answers surfaced. By the 55,000 accounts management thought were safe, you already expect about 3.5 colliding pairs, against the 0.7 a clean generator would give. The margin everyone treated as intact was already spent.

What actually caught it

None of this required a better cryptographer or a bigger model. It required a second measure that wouldn't defer to the first. The expert was wrong for a respectable reason and the lone model was wrong for a sloppy one, but they failed the same way: one frame, unchallenged, carried all the way to a signed-off number.

That is the failure a genuinely different reviewer catches and a smarter copy of the same reviewer does not. A model auditing its own work brings its own blind spot along for the audit. A model measuring with a different ruler brings a different question.

You can watch this exact run — the false inequality, the wrong ruler, the completion-of-squares proof — at shingik.ai.

Try it free — no signup. shingik.ai