When a council of AI models disagrees, the reflex is to take a vote. Three models said yes, two said no, ship the yes. It feels fair. It is the weakest thing you can do with those answers.
A vote quietly assumes two things that are almost never true. First, that every model in the room is equally reliable on this exact question. Second, that the right answer is even among the options being counted. Get either wrong and the tally doesn't find the truth. It launders whatever the loudest group already believed.
The default is a poll, and the poll is confident for the wrong reasons
This matters because the vote is the default. The consumer version of "ask several models" is a fixed panel, often clustered on two or three providers, with a synthesis step that either counts votes or blind-ranks the responses and ships the plurality. That design is most confident at the exact moment it should be least: when the wrong answer happens to have the numbers.
There is measurement on this. A benchmark called DeliberationBench put a weak five-model council through three naive aggregation protocols and found all three lost by roughly six to one to the boring baseline of just selecting the single best model's answer, while spending about 2.5 times the compute. Adding more voices and then counting them flat made the result worse and the bill higher.
And the vote is fragile in ways that have nothing to do with correctness. One paper ran 750 debates between three-model committees and found the "agreement" was mostly stagecraft. Shift the prompt's tone from friendly to hostile and full agreement moved 50.4 points. Strip a reading-order bias out of the judge and the verdict that "debate won" collapsed, 66 percent of the time, into 299 ties, with accuracy unchanged. If a show of hands moves fifty points on tone, the hands were never the point.
The headcount isn't the product
So here is the move. The value of a council is not how many answers you collect. It is two things, in order: seats independent enough that the right answer actually lands in the room, and a deciding layer that can tell which answer it is.
The headcount is not the product. The deciding layer is the product.
Weigh the voices, don't count them
Down-weight the member that is consistently wrong on this kind of question. A model that keeps missing medical dosing questions and keeps nailing contract language should not get one flat vote on both. Counting every seat equally throws away exactly the track record that makes a council worth running. A reliable member and a coin flip are not the same input, and a good deciding layer does not pretend they are.
Route the disagreement, don't average it
When two models split hard, that split is information about where the problem is hard. Averaging erases it. A good deciding layer treats a persistent, reasoned minority as a reason to look closer, not noise to outvote. Sometimes the lone dissenter is the only seat that read the question right, and a majority rule is the one rule guaranteed to miss it.
Verify before you decide
Recent work decomposes exactly when a debate beats a vote: the right answer has to be in the room, and the thing reading the room has to be able to pick it out, which means checking a candidate answer against the evidence before you commit to it. Verify first, decide second. A vote does it backwards. It commits to the plurality and never asks whether the plurality can survive a second look.
Keep the room from agreeing with itself
The input side still has to hold. An ICML 2026 paper finds a debate of models can quietly conform into a shared bias, and that heterogeneity across the seats is what suppresses it. If the panel is three near-siblings from the same model family, a unanimous answer tells you they were trained alike, not that they are right. Another ICML 2026 paper shows the inverse working in your favor: models red-teaming each other catch unsafe failure modes that a single model's own self-check sails right past.
Where we spend the design budget
This is why the deciding layer is where the work goes. Chairperson Synthesis exists to weigh divergence rather than tally it. It reads where the models split and why, instead of printing the majority and calling it consensus. The 200-plus models across providers are not a bigger menu for its own sake. They are how you draw seats that fail differently, so that a unanimous verdict actually means something. Red Team vs Blue Team forces the disagreement into the open, where the deciding layer has something real to read, instead of a quiet 3-to-2 that papers over a crack. And because you watch the debate stream in real time, you can see which model earned the verdict, not just which side had one more voice.
A council that settles its arguments by show of hands is barely a council. It is a poll with extra steps, and polls are confident for the same reason they are wrong: they count the room instead of reading it.
Count the moves. Weigh the voices.
Try it free — no signup. shingik.ai.