Mixture of Agents: When a Panel Beats a Genius

1. Proposers

Several different models (or the same model with different prompts/temperatures) each answer the task independently. Diversity is the fuel: different training data means different blind spots, so errors are less correlated.

2. Aggregation

An aggregator model receives all candidate answers alongside the original task and synthesizes a final response — keeping the best structure from one, the correct edge-case handling from another. In the original 2024 MoA paper, layered aggregation of open-source models outperformed GPT-4o on AlpacaEval 2.0.

3. Layers

Stack it: layer 2 proposers see layer 1’s outputs as reference material and improve on them. Two or three layers is the sweet spot; beyond that, returns flatten while cost keeps climbing linearly.

Why a panel can beat a stronger solo model

• Error decorrelation: one model’s hallucination is usually not shared by the others; the aggregator sees the disagreement and investigates.
• Coverage: a “42 head-to-head builds” style test rewards breadth — games need physics, art direction and state handling; different proposers nail different parts.
• Verification asymmetry: judging candidate answers is easier than generating them from scratch.

The honest trade-offs

• Cost: N proposers × L layers + aggregation ≈ 5–15× solo tokens.
• Latency: layers are sequential; each adds a full generation round.
• Aggregator ceiling: a weak aggregator can average away the one brilliant answer (“design by committee” failure).
• Vendor benchmarks: “our panel beat model X” claims deserve independent replication before you believe the marketing.

Try it yourself, cheaply

1. Send the same prompt to 3 models at temperature 0.8.
2. Paste all three answers into a 4th call: “Synthesize the best final answer; note where candidates disagree.”
3. Compare against any single answer. Disagreement notes alone often surface bugs you’d have shipped.