Vox Populi for Speculators

The wisdom-of-the-crowds thesis is rooted in an essay written by the English statistician Francis Galton in 1907. He observed a contest where eight hundred people guessed the weight of an ox. After gathering all of the estimates, Galton found that the median estimate was only nine pounds off from the actual weight of the ox. Galton urged future hosts of contests like the one he studied to share their results with scientists in order to advance the study of democratic judgments — an emphasis he underscored by titling his essay “Vox Populi.”

The wisdom-of-the-crowds theory is a powerful argument in favor of the brilliance of laypeople over supposed experts. The economist Friedrich Hayek developed a similar argument in his 1945 essay, “The Use of Knowledge in Society,” where he contended that buyers and sellers in markets were better stewards of the economy than Keynesian or Soviet bureaucrats. More than half a century later, journalist James Surowiecki published The Wisdom of Crowds book (2004), which popularized the theory by documenting the reliability of the “Ask the Audience” lifeline on Who Wants to Be a Millionaire?, as well as the supposed superiority of PPMs over other forms of political forecasting, like public opinion polls, in forecasting election results.

Although the wisdom-of-the-crowds notion persuasively counters the standing of authoritative experts, the whole idea rests — ironically — on the intervention of experts in at least two respects. First, the crowd does not magically deploy “wisdom” on its own; it must be rendered legible by an authority capable of determining what the crowd is actually saying. Second, its accuracy must be assessed against some independent standard. Galton didn’t believe the crowd was correct simply because it agreed with itself. Rather, he was impressed because its median estimate approximated the reading of a calibrated scale. Either way, the crowd’s apparent accuracy required Galton’s intervention. He collected and tabulated the guesses, calculated the average, and excluded thirteen guesses because their handwriting was illegible.

Proponents of PPMs routinely champion the idea of “putting your money where your mouth is” to challenge the incorrect predictions offered by public opinion pollsters and talking heads on cable news. While the image of people betting on their beliefs is enticing — and recalls the main thrust of the wisdom-of-the-crowds thesis, with a dash of Hayek — the act of betting in PPMs actually depends on the existence of a market infrastructure. Operators of Kalshi and Polymarket point to the structures of their markets to argue that prediction markets are not gambling markets — because wagers are made between bettors rather than between a bettor and a bookmaker — but this distinction functions less as a moral firewall against gambling than as a justification for treating wagers as information.

Making War the Jackpot

The problem of market infrastructure sprang to the fore recently when Polymarket declined to pay out bettors following the US incursion in Venezuela. Polymarket claimed that the operation did not count as an “invasion,” even though around one hundred people died and the Trump administration asserted that it was “running” Venezuela. This angered some bettors who maintained that the operation clearly met a reasonable definition of invasion.

When I interviewed an operator of a prediction market firm in Britain for my comparative research on the histories of PPMs in the United States and the UK, he said that — on top of the moral objections to hosting markets on wars — defining the occurrence of a “war” was difficult. For instance, does an initial attack qualify as the beginning of a war? Or must there be a declaration of a war from a government? Moreover, which sources does the market rely on to determine the occurrence of a war?

Whether American society should allow prediction markets for wars and other violent activities is an urgent moral question. Even setting that aside, shoddily designed market rules can undermine any claim that such markets generate reliable information.

The wisdom-of-the-crowds thesis may support the legitimacy of PPMs, but the economists who developed the first PPMs at the University of Iowa in 1988 based their approach not on that theory but on the work of experimental economists. Experimental economists — notably, the Nobel laureate in economics, Vernon L. Smith — disagree with Hayek that market actors are naturally rational. and they do not assume that crowds are always wise. Instead, they argue that rationality only emerges under carefully constructed market structures.

PPMs are organized as markets in which buyers and sellers simultaneously submit bids and offers, with prices changing continuously as new bets come in. Economists refer to this structure as a “double auction,” long considered the jewel of the experimental economics paradigm. But experimental economists emphasize that even these markets do not automatically or miraculously aggregate information efficiently — they need to be designed and managed. Tarek Mansour, one of the cofounders of Kalshi, has been explicit about the ambition behind prediction markets, saying that the goal is to “financialize everything and create a tradable asset out of difference of opinion.” But economic research shows that prediction markets will not create accurate forecasts for everything.

Spins of the Wheel, Signifying Nothing

There is another problem with using the wisdom-of-the-crowds thesis to justify the existence of the now-billion-dollar prediction market industry: for the crowd to be accurate, there needs to be some independent way of determining that it is accurate. When Galton calculated the median guess about the weight of the ox, he did not merely assume that the median guess was correct because it was the median. Rather, he was struck by the result because it so closely approximated the reading of the physical instrument used to measure the ox’s weight.

Historically, the Commodity Futures Trading Commission allowed firms like the Iowa Electronic Markets and PredictIt to operate only insofar as they facilitated academic studies of whether PPMs produced accurate political forecasts. PredictIt encountered regulatory trouble for allowing bets on how many times Trump would tweet in a given week, but these firms tended to confine themselves to election-related markets. The evidence of the long-term superiority of PPMs over other election forecasting methods, like public opinion polls and the aggregation techniques of Nate Silver, are mixed. However, given the nontrivial sample size of US elections, the wisdom of PPM traders can be productively compared to other predictive methods.

The current “financialize everything” ethos of Kalshi and Polymarket encourages the public to take it on faith that the crowd will be wise across an ever-expanding range of topics. Many modern prediction markets do not involve repeatable events like an election, nor do they offer alternative benchmarks with which to gauge the probability of a given outcome. Take, for example, a Polymarket market from 2024 on who would be identified as the real founder of Bitcoin in the documentary Money Electric. Despite robust betting and news articles about who the Polymarket bettors favored, Polymarket did not create a specific outcome for the person the documentary alleged to be the real Satoshi Nakomoto: Peter Todd. While some might argue that prediction markets are most useful precisely when no other methods of estimating probabilities exist, the inability to compare the derived probability to any independent measure renders that probability somewhat meaningless.

Even if, say, a market suggested a 60 percent likelihood that Todd would be named in the film, a strong probability would not imply the correctness of that probability. If I bet my friend that there was a 75 percent chance that a quarter landed on heads and I won, I could not reasonably claim that I was wise — I would just be making a bet for the fun of it. Winning a bet on a highly improbable outcome does not necessarily demonstrate wisdom; it simply shows that a bet happened to pay off. Without an independent way to check the result, the market’s probability is little more than a number attached to a price. However they are framed, prediction markets built on a “financialize everything” ethos are wagers with little forecasting utility.