Saturday 2 October 2004
ACT New Zealand Upper South Regional Conference The Wisdom Of Crowds - Roger Kerr Executive Director Christchurch New
Zealand Business Roundtable
THE WISDOM OF CROWDS Those of us who are engaged in public affairs may sometimes feel tempted to feel nostalgic for an
age – probably a mythical golden age – when informed ‘public opinion’ reigned unchallenged. The nineteenth century
sometimes seems like such an age: earnest, educated people read serious journals and wrote literate and thoughtful
letters to newspapers that were read by everyone who counted. They shared an unquestioned belief in science,
civilisation and progress that inspired effective movements for reform of all kinds. Wouldn’t public policy benefit if
we could rediscover their self-confidence, self-discipline and commitment?
Not according to James Surowiecki, a financial columnist for the New Yorker. The disconcerting message of his recent
book The Wisdom of Crowds is that consensus among the well-informed is quite likely to be wrong. In many areas, wisdom
is dispersed throughout society, often very thinly, but where it can be articulated, it is usually more reliable than
the wisdom of elites.
Surowiecki begins his book with an anecdote. It’s about Francis Galton, a British authority on statistics and the
science of heredity who was active about a century ago. Galton was interested in heredity because he believed that only
a few people had the necessary intelligence and ability to run society. It followed that only if power was concentrated
in their hands could society flourish.
In 1906 Galton‘s interest in breeding led him to an agricultural show, where he came across a weight-judging
competition. A fat ox was on display and members of the public could place wagers on its weight after it had been
slaughtered and dressed. Assuming that the average punter would be hopelessly wrong, Galton borrowed the several hundred
betting slips from the organisers and calculated the mean guess. To his surprise, it was 1,197 pounds: just one pound
short of the right answer.
Galton had stumbled on what Surowiecki calls a “simple but powerful truth … under the right circumstances, groups are
remarkably intelligent, and are often smarter than the smartest people in them … when our imperfect judgements are
aggregated in the right way, our collective intelligence is often excellent” (pp xiii–xiv).
It’s worth mentioning that this idea is an old one. Indeed, it was advanced by the ancient Greek philosopher Aristotle
in the fourth century BC. In his Politics, Aristotle defended a form of democracy by reference to the ‘wisdom of
crowds’:
It is possible that the many, no one of whom taken singly may be a good man, may yet taken all together be better than
the few, not individually but collectively … Each individual will be a worse judge than the experts, but when all work
together, they are better, or at any rate no worse.
Note that, for Aristotle as much as for Surowiecki, the wisdom of a crowd doesn’t depend on any member of the crowd
being wise; indeed, every member of the crowd could be hopelessly foolish. But in a crowd the mistakes are likely to run
in all directions and cancel one another out, leaving a sound judgment in the aggregate.
One anecdote does not establish a thesis, but the reader of the entire book is likely to be convinced that Surowiecki
is on to something. Here are a couple more examples.
In 1986, the American space shuttle Challenger exploded soon after it was launched, with the loss of all its crew.
Within minutes, the stock market reacted: the shares of the four companies that had most to do with the construction of
the shuttle had all lost value. But one of them, Thiokol, the manufacturer of the solid-fuel booster rocket, lost more
ground than the others. Moreover, the next day, when the shares of the other three companies began to recover, Thiokol’s
continued to fall. Six months later, the official enquiry confirmed what the stock market (but no one else) apparently
knew from the start: that the seals on the booster rocket were defective and had caused the explosion. An academic study
of the unique, almost immediate and – as it happened – well-founded collapse of Thiokol’s stock turned up no evidence to
support likely explanations, such as media speculation or Wall Street hype. In the end, the researchers concluded that
there must have been some insider dealing, although they couldn’t prove it. But Surowiecki thinks the most likely
explanation is that it is another example of the wisdom of crowds.
The other anecdote I’ll mention is closer to home but perhaps more conclusive. In the television quiz show Who Wants to
be a Millionaire? contestants stumped by a question can enlist the aid of friends or relatives chosen on the basis that
they think they are smarter or better informed than themselves. They can also opt to ask the studio audience. The
‘experts’ turned out to be right 65 percent of the time. But the members of the audience – supplying answers anonymously
and independently of one another – were collectively right 91 percent of the time. If you ever appear on that show, you
may do better to rely on the crowd than on your smart friends!
Clearly, we are happy to accept the intuitions of Surowiecki’s thesis in fashioning some of our social arrangements.
The most obvious in the modern world is democratic politics. We rely on a universal, unweighted one person–one vote
franchise to choose our governments. The merits of that form of democracy were by no means so obvious historically;
Aristotle, after all, was challenging Plato’s advocacy of rule by a philosopher king. Sometimes, as with Lee Kwan Yew,
such rule can be benign, indeed inspired. But the lesson of the twentieth century is that for every Lee Kwan Yew there
are a dozen or more Joseph Stalins, Idi Amins and Robert Mugabes.
In choosing a political system, as with other public policy choices, what matters is the performance of alternative
arrangements on average and over time. It is unwise to bet against the odds. Democracy is not perfect but we put faith
in the wisdom of crowds for choosing our governments on the basis that democracy is better than any alternative. Looking
back on the choices of the New Zealand electorate in the past 25 years, I find it hard to argue that that faith has been
grossly misplaced.
An immediate objection to Surowiecki’s thesis is that there is plenty of counter-evidence showing the folly of crowds.
The book openly acknowledges this evidence. Witch-hunts, the rise of Adolph Hitler, and stock market bubbles could all
reasonably be cited as examples of crowds, sometimes entire nations, taking collective leave of their senses. Indeed,
the assumption that crowds are more likely to get things wrong than right seems to underlie many of our familiar
arrangements and practices. Where knowledge is concerned, we have entrenched divisions between experts and laypeople.
While we believe in democracy, we usually stick with the indirect, representative kind: the idea is that we are better
to let the decisions be made by an elected elite that has some incentive to become well-informed about issues, and let
the people as a whole judge the results at the next election. It follows that, if Surowiecki is right and that “under
the right circumstances”, as he puts it, the amateur masses may be wiser than the professional elites, some of those
practices and arrangements should be reviewed. Naturally I am interested in the implications of Surowiecki’s thesis for
public policy and I consider some of these later on.
Under what conditions, then, are crowds more likely to be wise than foolish? Surowiecki’s thesis naturally depends very
greatly on the soundness of this part of his analysis. He says that crowds are likely to be wise when four conditions
are met: diversity of opinion (everyone has a bit of private information, however small); independence (people’s
opinions are not influenced by those of others); decentralisation (people can specialise and draw on local knowledge);
and aggregation (there must be a mechanism for turning individuals’ private judgments into collective decisions).
In the economic sphere we can note at once a familiar example of ‘crowd wisdom’ as Surowiecki defines it: the
marketplace. The market is driven by diverse tastes and preferences; consumers normally make private judgments about
those tastes and preferences and can act on them; the market, being driven by individual choice, is as decentralised as
any institution can possibly be; and consumers’ choices are continually aggregated by the price mechanism.
Today it is uncontroversial to argue that, like democracy, the free market economy outperforms alternative systems.
Again this was not always the case. Serious scholars in the twentieth century, not to mention political demagogues,
advocated central planning of the economy. Putting resource allocation decisions in the hands of ‘experts’, it was
argued, would lead to more growth and prosperity than leaving them to the spontaneous and unpredictable workings of
competitive markets.
The collapse of socialist central planning has dispelled this illusion for most. Central planners have neither the
information nor the incentives to make good decisions compared with the knowledge and incentives of decentralised
economic actors, ie the people at large. Yet we continue to see innumerable examples of government interventions based
on the premise that politicians and bureaucratic experts in Wellington can spend people’s money and run their lives
better than they can themselves.
A more specific economic illustration of the wisdom of crowds is the operation of financial markets. The prices of
financial assets (such as bonds, stocks and currencies) are determined through competitive trading. A great deal of
research indicates that these markets are very efficient in the sense that they reflect all publicly available
information about the fundamental value of these assets. Unless investors or traders have information that is not
publicly available, it is very difficult for them to ‘beat the market’ – to do better than the collective assessment of
the vast numbers of people dealing in financial markets every day.
Nevertheless, crowd wisdom is not infallible, and Surowiecki devotes much of his book to showing how the absence of any
of his four conditions diminishes it. Diversity is especially important to correct the ‘groupthink’ that can so easily
creep up on teams of experts and lead them astray. Experiments have shown that groups of people of mixed ability are
often better at solving problems than groups comprising only smart people. The problem seems to be that experts tend to
share, but also to exclude, a good deal of knowledge as a result of their training, whereas less smart types may bring
useful information that experts have collectively forgotten or never had. (Surowiecki cites by way of examples of
failures of ‘expert’ wisdom the persistently poor performance of professional financial advisers and the failure in 2003
of NASA to register fears among some NASA staff about damage to the space shuttle Columbia, which eventually caused the
shuttle to break up when it re-entered the earth’s atmosphere.) The point is not that expertise is to be avoided, but
that experts can often learn from lay people, so the best decisions are likely to be a joint product of both types – if
that can be arranged.
Independence – each member of the crowd should form a judgment without being directly influenced by the other members –
is an equally important condition for the wisdom of crowds. Its absence can explain the examples of mass folly we
mentioned earlier. A stock market bubble can occur when investors stop making independent judgments and become overly
influenced by what other investors think. So much is clear. But independence also tells against consensus. The wisdom of
crowds is emphatically not something that emerges from negotiation among the members of a crowd. Surowiecki cites
research showing that group discussion tends to make the group and its individual members more extreme in their views
than they were originally (perhaps special interest lobbies bear this out).
This idea too has a good pedigree in classical democratic theory. The eighteenth-century philosopher Rousseau argued in
The Social Contract that in making decisions the members of the sovereign assembly should have no communication with one
another and “each citizen should think only his own thoughts”. Otherwise, Rousseau argued, the assembly would become
corrupted with special interests and would lose sight of its shared interests. Indeed, we recognise the political
importance of independence of judgment in the secret ballot, which protects each voter from undue pressure from other
voters: we want voters to make their own judgment, not one that reflects the influence of the last person they spoke to.
Decentralisation is probably the least controversial condition for the emergence of the wisdom of crowds, since we
obviously want every member of the crowd to contribute to its collective judgment, and decentralisation encourages
specialisation and diversity. As for aggregation, the last condition, Surowiecki uses a topical example to illustrate
its indispensability: the activities of America’s intelligence agencies. As we know, some agents were trying to warn
about the possibility, even the likelihood, of an imminent terrorist attack on American soil. But as we also know, the
agencies had not worked out a way of bringing together the collective view of their staff, and so the warnings did not
register where they should have.
I said earlier that I was interested in the public policy implications of the ‘wisdom of the crowd’ thesis. I’ve
already cited the market as an area in which we generally accept that the crowd is wise or, at the very least, less
foolish than any other source of judgment. The New Zealand Business Roundtable is, of course, well-known for its
advocacy of markets, and of extending market processes into certain areas presently dominated by the state, such as
health and education, where evidence of the wisdom of bureaucratic elites is indeed hard to come by. These sectors,
which are vital to economic and social progress, remain largely nationalised and centrally planned; we are still
operating, so to speak, behind the Berlin Wall. In education, for example, school zoning, an extensive national
curriculum, teacher employment arrangements and the absence of a level playing field with the private sector heavily
constrain decentralised decisions by individual parents. Not surprisingly, the frustrations of parents, teachers and the
community at large run high. Worldwide trends towards school autonomy, parental choice and a greater role for the
private sector suggest that the days of the Berlin Wall are numbered. The shape of the system would then be determined
less by education planners and more by the cumulative decisions of individual families.
But the more original public policy implications of Surowiecki’s thesis are for our democratic political system.
Representative democracy, through the universal and secret ballot, is, on the analysis offered in the book, a way of
eliciting the wisdom of crowds in making collective decisions. But it is not the only way. Another, which Surowiecki
does not explore, is to make more use of direct democracy, that is, referenda.
One of the arguments for representative or indirect democracy, as I mentioned earlier, is that our elected
representatives are in a position to become informed about issues and that the general public’s role should therefore be
confined to judging a government’s overall performance. However, this argument is open to Surowiecki’s objection that
groups of experts are liable to become impervious to relevant information that non-experts can supply. Governments are,
of course, far from insulated from public opinion. But one of the causes of the malaise that infects Western democracy
is the rise of politics as a profession, as a result of which politicians as a whole, whatever party they belong to,
seem to have more in common with one another than with their electorates, and, along with their advisers and top public
servants, can with some justice be said to constitute a ‘political class’ that is vulnerable to groupthink. That is to
say, representative assemblies are becoming less truly representative as they lose diversity. Referenda are a way of
counteracting that tendency and getting more diversity and hence more relevant information into the political system.
They also enhance decentralisation by allowing members of the public to express views on particular issues, which they
cannot easily do in general elections when they typically vote for a political party or coalition.
Thus, contrary to the 1986 Royal Commission on the electoral system, I see merit in greater use of referenda for making
those decisions that have to be made collectively. It is surely high time, for example, that the country was given
another say on whether to retain the mixed member proportional (MMP) electoral system. Even the prime minister has said
that should happen, but has to date argued that it is too soon to have another referendum. That argument is becoming
less and less credible. In my view the nuclear ship visit issue is one that would be well-suited to a referendum; I, for
one, would be happy to accept ‘crowd wisdom’ in weighing up competing claims. And in a report which the Business
Roundtable is releasing this month, the case is made for amending the Fiscal Responsibility Act 1994 to require voter
approval via referenda for new taxes or tax rate increases.
An intriguing proposal that Surowiecki discusses is the possibility of a policy analysis market. This idea seems to
have grown out of the Iowa Electronic Markets (IEM) project that was set up in 1988 as a competitive alternative to
opinion polls. The IEM is simply a market for betting on the outcome of elections: any member of the public can “buy and
sell futures ‘contracts’ based on how they think a given candidate will do in an upcoming election” (p 17). The
interesting thing is that between 1988 and 2000 the IEM generally outperformed opinion polls, even though only a few
hundred people participated in them. Note the difference between a betting market and an opinion poll. In the betting
market, people are making bets on how other people will vote and backing their judgment with their own money. In a poll,
people say how they themselves intend to vote, but there are no sanctions if they don’t give a truthful answer or if
they change their minds later. The incentive to get a forecast right is obviously much greater in a betting market.
Some private corporations have tried to tap the wisdom of their employees by setting up ‘decision markets’, based on
the same principle as the IEM, on the likely success of their new products. Those cited by Surowiecki –
Hewlett-Packard’s on new printer sales, and Innocentive’s on which new drugs were likely to win approval from the US
Food and Drug Administration – have been remarkably accurate. How much easier for humble employees to contribute their
knowledge, and to gain from doing so, through an anonymous market rather than through some consultation committee, with
its built-in biases, managers unwilling to register inconvenient information, and employees afraid of the consequences
of speaking up.
A leading advocate of decision markets is Robin Hanson, of George Mason University in the United States. In an article
written in 1999, he says: … we suffer from a serious failure to share information. The so-called Information Revolution
has greatly improved our ability to find out what others have said. However, it has done much less to improve our
ability to find out what other people know … Speculative markets are a neglected way to help us find out what people
know. Such markets … have many advantages over standard institutions for information aggregation, such as news media,
peer review, trials, and opinion polls. Speculative markets are decentralised and relatively egalitarian, and can offer
direct, concise, timely, and precise estimates in answer to questions we pose. Just as self-interest motivates us to
influence the market price of a good by buying it or not buying it, so the gains from betting can motivate us to
contribute accurate and relevant information to a collective decision, and more reliably so than through mechanisms
where no such incentives exist.
How then would a policy analysis market work? The term comes from one such market that was set up by the Pentagon in
2003 to provide intelligence on the Middle East. The general public would be invited to make bets on possible events in
the region, with a view to allowing policy-makers to bypass the vested interests of intelligence agencies in their pet
theories and analyses and get some new, informal but reliable information into the policy-making arena. But it was not
to be. Congressional pressure forced the Pentagon to scrap the idea on the grounds that it was morally wrong to
encourage people to profit from bets on terrorist outrages, even though it was not on that sort of question that the
policy market was expected to focus. Yet would we not approve if a government agent profited in career terms as a result
of accurately forecasting the attacks of 9/11? (Surowiecki, who is naturally a fan of policy analysis markets, passes on
the good news that in March 2004 NetExchange, the company that set up the market for the Pentagon, announced that it
would open a new one, without government involvement, devoted to forecasting broad economic and military trends in the
Middle East. Surowiecki also cautions that the events wagered on would have to be appropriately framed to allow a
reliable market to emerge. They would have to deal with the problem of a wager becoming self-defeating; for example, a
forecast assassination could be successfully anticipated and forestalled. But that would only confirm the reliability
and indeed the very usefulness of such markets.)
It’s interesting to speculate on how policy analysis markets could be developed to cover the entire field of policy. A
private organisation – a think tank, a newspaper, a television or radio company, even a public opinion polling company –
could set up a betting market on, say, whether our current Labour-led government will, on present policies, achieve its
‘top priority’ objective of returning New Zealand to the top half of the Organisation for Economic Cooperation and
Development (OECD) income rankings by some specified date. The qualification ‘on present policies’ would be necessary
because the government could react to a betting market forecast of failure by changing its policies to ones more likely
to succeed. And that is exactly how such a market could help improve public policy. It could be expected to be more
reliable, and hence more heeded, than opinion polls, which are vulnerable to politicians’ growing skill in manipulating
voter apathy, ignorance, and cynicism. I’m tempted to keep speculating in this vein. What, I wonder, would a betting
market have forecast on whether the Coalition that invaded Iraq would find weapons of mass destruction, and with what
effect? But rather than continue along these lines, let me sum up.
We live in an age of mass higher education; every topic one can think of can be studied somewhere for a qualification,
so that it has its official ‘experts’. One thing I’ve discovered is that people with educational qualifications
certainly tend to believe that they know more than laypersons, and they also tend not to be backward in promoting new
legislation that embodies their expertise, regardless of the cost. So whatever its benefits, expertise needs to be
accountable. The best mechanism for making it accountable is the market, where the consumer is sovereign but also
ensures that any benefits of expertise are recognised and rewarded.
Where markets are not workable, as with pure public goods, the elitism of ordinary politics could benefit from the
additional accountability provided by the referendum. And betting markets in policy areas could amount to a sort of
continuous popular referendum – a more reliable and stable one than that offered by volatile opinion polls, as well as
being a lot of fun.
ENDS