Limitations of wisdom


Panos Ipeirotis published a nice analysis of the independence assumption of Surowiecki’s Wisdom of Crowds theory. In short, he finds that in some cases independence is necessary, in some cases it seems that some information leakage doesn’t hurt, and in another class of circumstances, pooling information leads to reliably better outcomes than independent guessing.

What’s going on?

Panos uses example of prediction markets to show that partial information sharing (in the form of the fluctuating market price) does not detract from the effectiveness of this approach. My guess is that markets, operating in a “normal” regime, represent an aggregation of different expectations of future performance, which leads to different behaviors in the presence of a shared current information. When everybody’s expectations align, the market doesn’t do well: Bubbles and runs on banks are examples of this.

The other example Panos cites where collective behavior outperforms independent assessments represents a highly-constrained set of options: guessing the outcome of a four-sided die given that different people have different partial information. The relatively simple space of outcomes makes it possible  to apply inference to combine partial information. In more complex situations where information cannot be decomposed and recombined, we should see the classical behavior.

I am no mathematician and cannot even begin to articulate a proof of any of this, but it seems plausible to me that the Wisdom of Crowds approach works best with complex, multi-dimensional spaces; when the space is constrained, other strategies may become more effective.