Social scientists have always been fascinated by crowds. From guessing the weight of a cow to identifying which company built the faulty part that led to the Challenger space shuttle disaster, the many have often been able to outguess the expert few. Crowd wisdom is often cited as the justification for the idea of efficient asset markets - many investors, each weighing in with their buying and selling decisions, should combine to produce the optimal forecast of what a stock or a bond is really worth. Or so the story goes.
But there is danger in relying on crowds to make decisions. Under certain circumstances, group wisdom can break down and become madness, such as when Reddit users tried to identify the Boston Marathon bomber, and ended up accusing the wrong guy. Many believe asset-market bubbles are also examples of crowds gone mad.
Why are crowds sometimes wise and sometimes mad? Social scientists already have a rough idea of the general answer to that question. Crowd wisdom works because people's mistakes are haphazard and uncorrelated. Everyone's guess is a combination of signal and noise - we have some idea of the real weight of a cow, or the real value of a stock, but we also have our own wrong ideas and preconceptions and irrationalities. But because my errors are not the same as yours, when we combine our guesses, the true knowledge shines through while the random errors tend to cancel out.
But when people in a crowd communicate, their mistakes are no longer uncorrelated. When one person's misjudgments influence another person's thinking, the errors can snowball and wreck the whole forecast. Any number of studies will confirm the general principle - once people start talking, arguing and persuading one another, crowds turn into herds and the magic disappears.
Why do people influence one another? There could be all sorts of reasons, both rational and irrational. People might simply have an instinct to copy other people's actions, or take their word as gospel - the old saw of "if you read it, it must be true". Economists have built elaborate models of how rational herd behaviour might cause bubbles and crashes. Alternatively, copying what other people do might be perfectly rational in many situations - if you see everyone in a cafe suddenly run for the exits, it might be a good idea to follow as quickly as possible.
The difficulty comes in applying this insight to real-world problems. In real-life situations such as investing, it is a certainty that most people have received some kind of information from others - stock tips, Bloomberg News articles, investing advice, TV shows and so on. The question is how much people actually heed others' opinions, as opposed to simply taking in factual information.
A team of researchers from the Massachusetts Institute of Technology's Sloan Neuroeconomics Lab may have found a new way to identify herd behaviour before it strikes. In their paper, researchers Drazen Prelec, H. Sebastian Seung and John McCoy asked forecasters a new and unusual question. In addition to simply asking people for their guesses, they asked what people thought others would guess.
If herd behaviour was present, some people would know it, and would be contrarians - they would guess something different from what they thought other people would say. The researchers found that the forecasts that received the most contrarian support - the guesses that people picked even though they thought others would guess differently - tended to be the right ones. They found that these forecasts, which they labelled the "surprisingly popular" options, tended to outperform standard crowd averages in a number of applications, with error rates more than 20 per cent lower.
Herd behaviour is not the only reason this method might work. Another possibility is the Dunning-Kruger effect - the fact that ignorant people also tend to be ignorant of their own ignorance, while knowledgeable people know they are better informed. This is closer to the explanation that the researchers give for their result. But since herd behaviour is the best-known force that breaks down the wisdom of crowds, it seems likely to me that any method that improves so much on traditional crowd-based forecasting does so by partially counteracting the herd behaviour's effect.
In any case, this method obviously has some very important potential applications for finance. Hedge funds or other investors could poll investors, or their own analysts, using the researchers' method, and potentially beat the market. The Federal Reserve could use large groups of forecasters to identify when asset bubbles were happening, and try to pop the bubbles with interest rate hikes or other policy measures. And the government might loosen restrictions on short-sellers, who tend to be contrarians.
In the search for the ultimate forecast, the wisdom of crowds might turn out to be very good, but not quite the best.
BLOOMBERG VIEW
