Silver intelligently combines multiple polls to make more accurate predictions than are usually achieved by any one individual pollster. On one hand, the math of this is irresistible – in the real world, voting models often work. On the other hand, it would be pretty uncomfortable for a pollster to combine his own results with various competitive poll results to achieve equivalent accuracy (or at least to do so transparently). So, the pollsters do all the tedious work to collect and analyze the data, and then Nate Silver comes along and creates all this value with it in a way that is hard for the pollsters to duplicate. You can see why this situation might upset the pollsters.
In every industry that combines data collection with analysis, there is an endless battle between the data collectors and the analysts. The data collectors bear the hard costs – people, office space, telecommunications, travel budgets, etc. – that are required for interviewing people, visiting stores, and so forth. Their nightmare world is to become commodity data collectors paid for their costs plus a small margin set by competitive bidding. Their typical defenses are to attempt : (1) to build proprietary methods for collecting superior data, or equivalent data at much lower cost, and (2) to integrate the analysis and the data into a single product, and forbid by contract the paying client from using this for other purposes. The analysts, on the other hand, want to have an open market in commoditized data and compete on analytical capability.
The pattern Manzi outlines is at work in the investment world as well. On one level, conventional equity research, like polling data, is now subject to aggregation and analysis; services like Investars and Reuters' Starmine offer average ratings for equities as well as ranking analysts by the performance of past ratings on given stocks and sectors. Traditional fundamental research has become commoditized to a degree, as the research itself, like the material company information of which it's composed, can no longer be provided selectively to favored clients. More broadly, active fund management itself is giving way to indexing, in large part through ETFs. Suzanne Duncan of the IBM Institute for Business Value has forecast,on the basis of a study polling financial executives and their clients, that within twenty years, 85--90% of assets under management will be invested in passive instruments such as index funds. (Not coincidentally, the study also found that only 10% of hedge funds actually produce any alpha, i.e. earn their exorbitant fees. The same is doubtless true of managed mutual funds.)
