Qualifying the Wisdom of Crowds

For Sunday fun, a jump into the wisdom/unwisdom of crowds discussion. Re the oft-cited fact that when a roomful of people are each asked to guess the number of jellybeans in a jar, the average answer is usually spot-on, Ed Yong notes:

But all of these examples are somewhat artificial, because they involve decisions that are made in a social vacuum. Indeed, James Surowiecki,  author of The Wisdom of Crowds, argued that wise crowds are ones where “people’s opinions aren’t determined by the opinions of those around them.” That rarely happens. From votes in elections, to votes on social media sites, people see what others around them are doing or intend to do. We actively seek out what others are saying, and we have a natural tendency to emulate successful and prominent individuals. So what happens to the wisdom of the crowd when the crowd talks to one another?

Andrew King from the Royal Veterinary College found that it falls apart, but only in certain circumstances. At his university open day, he asked 82 people to guess the number of sweets in a jar. If they made their guesses without any extra information, the wisdom of the crowd prevailed. The crowd’s median guess was 751.* The actual number of sweets was… 752.

This collective accuracy collapsed if King told different groups of volunteers about what their peers had guessed. If they knew about the previous guess, a random earlier guess or the average of all the earlier guesses, they overestimated the number of sweets. Their median guesses ranged from 882 to 1109. King likens this effect to real-world situations where people collectively drive the prices of items above their value and create economic bubbles. It’s what happened to create the recent US/British housing market crash or, more historically, the tulip mania of 17^th century Holland.

 Kevin Drum pounces:

This is why I've always been skeptical of the whole wisdom of crowds thing. You can make up all sorts of contrived situations where it works, but there aren't very many examples from real life where people don't communicate in one form or another and form feedback loops. We're a social species, and lots of communication is very much the norm, not the exception.

I commented on Drum's site:

I find myself thinking about product ratings. In some cases, as in a popular restaurant that gets more or less the same Zagat rating year by year, reviewers may be influenced by past opinion. But in others, e..g.,when you're moved to review a hotel on TripAdvisor because of a really good or really bad experience - -the crowd sourcing may really work here. I suspect that many or most review sites are hybrids -- many folks will weigh in with their own opinions because of a strong experience, while others will either first use the site to make a decision and then weigh in with their own,  or read the site for pleasure and then weigh in -- as a literary-minded person might do on a given book on, say, Amazon.  But if you're buying, say, a dehumidifier, and you select a product w/ 300 reviews on Amazon, chances are you'll be able to access the wisdom of crowds. 

I suspect that voting is something of a hybrid, too. Drum's skepticism is overstated, if it's meant to apply to real-world electorates. I think that there is an element of constructive crowd-sourcing in votes, and that electorates get things right more often than not, given the choices they're presented with or which they in some partial sense present themselves with. First of all, even in the case of the knowledge-tainted jellybean test, the estimates are not that terrible, though they're biased in one direction.  The average group guess is somewhere in the range of 50% overstated. I suspect that I, for one, would be perfectly capable of over- or underestimating the number of jellybeans in a jar by a factor of five. Second, the information that King gave his students to play with is not analogous to the information that voters access or fail to access. Third, only a small portion of any given electorate "swings," and most swing voters are low-information voters. While many of them may be moved by pockets of social influence, I suspect that a preponderance are moved either by the most salient facts about the current state of the community, e.g., the state of the economy or the course of a war, or by gut perceptions about the rival candidates that may collectively mesh something like individuals' tiny unconscious calculations about the jellybeans in the jar.

However those two factors tilt the field, the "pockets of social influence," i.e., the rival campaigns and their legions of official and self-appointed proxies, are left to ply their wares among those who by definition are capable of swinging either way. When rival social factors are in play, the wisdom of crowds may work on them.  So, while social factors permeate the process, I think that to the extent an election is free, the wisdom of crowds still has some room to run.