Smart nodes don't make a wise network

The question

Agent systems use adversarial review: some agents do work and make assertions, others critique that work and argue about whether it's true. This mirrors human collective sensemaking. On any social platform there are assertions and people dunking on them, all part of a collective process driven by individual beliefs.

What does that say about sensemaking and AI design? What stays constant between humans and agents as nodes on a dynamic network of information processors trying to understand reality?

The core insight: truth-tracking belongs to the network, not the node

Adversarial review in agent systems and the dunk-fest on social media are versions of the same process: distributed error correction among nodes that each see only part of reality. Comparing them also shows why one sometimes converges on truth and the other often doesn't.

No single node, human or agent, has privileged access to reality. Each has a partial view, a set of priors that filters what it notices, and a tendency to defend what it already said. Individually, nodes are biased.

Hugo Mercier and Dan Sperber's argumentative theory of reasoning holds that human reasoning evolved for this social setup, not for solitary truth-finding. People are poor at critiquing their own claims (myside bias is strong) but good at spotting flaws in other people's. Groups that argue solve logic puzzles individuals fail at. Reasoning works as a two-role system: biased advocates plus sharper evaluators.

That is almost exactly the architecture of generator and critic agents. You don't fix the generator's bias by making it unbiased. You fix it by putting it in a structure where someone else is motivated to find the holes. Science, courts, peer review and open-source code review all work this way: institutions that produce reliable knowledge out of unreliable individuals.

The loop

Both substrates run the same four steps:

Step	Humans on a platform	Agents in a harness
Assert	A post, a hot take, a confident claim	A generator agent's answer, plan or code
Challenge	The reply, the quote-post, the dunk	A critic agent probing for errors and gaps
Adjudicate	Mostly attention (likes, ranking). Sometimes an editor, a court, an experiment	A judge agent, a test suite, or a human reviewer
Record	The feed, which forgets by tomorrow, or science's literature, which accumulates	A shared log of claims, objections and resolutions, if you build one

Adjudication is where the process is won or lost. Each round's record becomes the next round's priors.

What's constant across humans and agents as nodes

These hold whether the node is a person, an LLM agent, or a unit in some larger system:

1. Partial view. A context window and a lived experience are both bounded. Nobody holds the whole picture.
2. Priors shape perception. Training and system prompts work for an agent the way upbringing, identity and expertise work for a person: they decide what counts as salient or plausible.
3. Making claims is cheap; checking them is expensive. Brandolini's law: refuting nonsense takes far more energy than producing it. In AI the asymmetry can run the other way and makes review useful: for many tasks, checking an answer is easier than generating it. Adversarial review works best where verification is cheap.
4. Incentives shape output. Humans post for status and belonging. Agents drift toward whatever they're rewarded for, which is where sycophancy comes from. Neither kind of node purely seeks truth.
5. Errors are correlated. The Condorcet jury theorem says crowds beat individuals, but only if their errors are independent. A thousand people repeating one influencer don't count as a thousand opinions. Five critic agents on the same base model with the same context don't either. Agreement between them can be an echo, not confirmation.
6. Topology matters. Kevin Zollman showed that less-connected networks of scientists sometimes reach the truth more reliably. Sparse connections keep minority hypotheses alive long enough to be tested; full connectivity causes premature convergence. Social media is close to fully connected, with a recommendation engine pushing on top.

Why social media mostly fails at sensemaking

The difference isn't the presence of argument. It's what the arena rewards. Productive adversarial systems share features social media lacks:

1. Grounding. Science has experiments and code has tests: something outside the argument can say "you're wrong." Social media disputes rarely touch ground truth. They resolve by attention.
2. An adjudicator with standards. Courts have judges and rules of evidence; peer review has editors. On social platforms the "judge" is the engagement algorithm, which rewards conflict, not resolution.
3. Critics rewarded for accuracy, not performance. A dunk is mostly an identity signal to your own tribe. It works on the speaker's standing, not the claim's validity. A good critic is rewarded for finding real flaws and ideally penalized for false alarms.
4. Memory. Science builds a cumulative record. Social media resets every news cycle, so the same arguments get relitigated forever.

Adversarial process alone isn't truth-seeking. It's an amplifier that serves whatever objective the system optimizes. Point it at accuracy and it's error correction. Point it at engagement and it's polarization.

This is also the open question in AI safety's debate proposal (Irving, Christiano & Amodei, 2018): debate converges on truth only if honest arguments are easier to defend than dishonest ones, and that depends on a competent judge and checkable claims.

What this means for designing agent systems

Building multi-agent systems is institutional design, a scaled-down version of the problem humanity solved with science, law and markets:

1. Engineer independence. Different models, different context slices, critics who don't see the generator's reasoning so they can't anchor on it. Diversity is the resource that makes the crowd wise.
2. Ground wherever you can. Prefer critiques that cite an executable check, a source or a test result over critiques that only argue. Ungrounded agent debate drifts into the same rhetoric contest as Twitter.
3. Calibrate the critic's incentives. A critic told to "find problems" invents them (contrarian theater). One that's too agreeable rubber-stamps. Ask for severity and confidence, and track which critiques turned out right.
4. Adjudicate, then stop. A judge with explicit standards of evidence, and a stopping rule. Otherwise you get infinite relitigation, like the timeline.
5. Keep a persistent record. A shared log of claims, challenges and resolutions, like Wikipedia's talk pages or a lab notebook, so the system learns instead of recycling arguments.
6. Distrust fast unanimity. Unanimous, instant agreement among similar agents is a warning sign, not a success.

The bigger picture

Every node is a prediction engine trying to reduce its own surprise, and collective intelligence emerges from how those engines are connected and what they're rewarded for. Individual rationality matters less than epistemic architecture. A network of clever nodes with bad topology and bad incentives will be confidently wrong together. A network of mediocre nodes with good structure can be reliably right.

The twist: AI agents are becoming nodes in the human network too. They can flood it with cheap claims, or they can make verification cheap at scale. Whether AI makes collective sensemaking better or worse may depend less on how smart the models are than on which of those two roles we design them into.

Smart nodes don't guarantee a wise network. Wiring and incentives do.

Sources

• Hugo Mercier and Dan Sperber, The Enigma of Reason (2017). The argumentative theory of reasoning.
• Kevin Zollman, "The Communication Structure of Epistemic Communities," Philosophy of Science (2007).
• Geoffrey Irving, Paul Christiano, and Dario Amodei, "AI Safety via Debate" (2018).
• Marquis de Condorcet, Essay on the Application of Analysis to the Probability of Majority Decisions (1785). The jury theorem.
• Alberto Brandolini, the "bullshit asymmetry principle" (2013).