The anguish of the data collectors

Jim Manzi views the dispute between poll analyst Nate Silver and pollster John Zogby in economic terms:

Silver intelligently combines multiple polls to make more accurate predictions than are usually achieved by any one individual pollster. On one hand, the math of this is irresistible – in the real world, voting models often work. On the other hand, it would be pretty uncomfortable for a pollster to combine his own results with various competitive poll results to achieve equivalent accuracy (or at least to do so transparently). So, the pollsters do all the tedious work to collect and analyze the data, and then Nate Silver comes along and creates all this value with it in a way that is hard for the pollsters to duplicate. You can see why this situation might upset the pollsters.

In every industry that combines data collection with analysis, there is an endless battle between the data collectors and the analysts. The data collectors bear the hard costs – people, office space, telecommunications, travel budgets, etc. – that are required for interviewing people, visiting stores, and so forth. Their nightmare world is to become commodity data collectors paid for their costs plus a small margin set by competitive bidding. Their typical defenses are to attempt : (1) to build proprietary methods for collecting superior data, or equivalent data at much lower cost, and (2) to integrate the analysis and the data into a single product, and forbid by contract the paying client from using this for other purposes. The analysts, on the other hand, want to have an open market in commoditized data and compete on analytical capability.

The pattern Manzi outlines is at work in the investment world as well.  On one level, conventional equity research, like polling data, is now subject to aggregation and analysis; services like Investars and Reuters' Starmine offer average ratings for equities as well as ranking analysts by the performance of past ratings on given stocks and sectors. Traditional fundamental research has become commoditized to a degree, as the research itself, like the material company information of which it's composed, can no longer be provided selectively to favored clients.  More broadly, active fund management itself is giving way to indexing, in large part through ETFs. Suzanne Duncan of the IBM Institute for Business Value has forecast,on the basis of a study polling financial executives and their clients, that within twenty years, 85--90% of assets under management will be invested in passive instruments such as index funds. (Not coincidentally, the study also found that only 10% of hedge funds actually produce any alpha, i.e. earn their exorbitant fees. The same is doubtless true of managed mutual funds.)

The end of money management as we know it?

At Investorside's Independents' Day conference of independent research providers on April 8, Suzanne Duncan of the IBM Institute for Business Value reported some startling results from a study of the current state and likely future of the financial industry. Undertaken jointly with the Economist Intelligence Unit and the CFA Institute, and surveying 848 financial markets executives worldwide and 107 of their corporate clients, the study found:

• 70% of the institutions' corporate clients said that their providers put their own interest first, not their clients'. 60% of industry executives surveyed admitted to the same thing.
  
  
• Only 10% of hedge funds deliver alpha -- that is, do what they're paid to do.
  
  
• Within twenty years, 85--90% of assets under management will be invested in passive instruments such as index funds.

That last forecast (if correct) means pretty much that the money management business as we know it is going to evaporate --though Ms. Duncan, asked what the near-end of active investing meant for the analysts filling the room, did suggest that there would be demand for research (and presumably advice) in asset allocation.

I asked Ms. Duncan: if 85-90% of funds are passively managed, who's going to set prices? She said that academics who have made even more radical forecasts have suggested that price discovery can be adequately handled by those managing just 5% or even less of total assets.

Personally, I'm all for index investing. I've always suspected that 90% of hedge funds were frauds, and that only the best-endowed institutional investors, such as the the top Ivies, had both the acumen and the financial clout to get into the relative handful of truly effective hedge funds under favorable terms. But as Jonathan Clements, the former Wall Street Journal personal finance columnist and evangelist for index investing (now Director of Financial Guidance for Citi's fee-only MyFi advisory service) once wrote, those of us who invest in index funds and other passive vehicles are parasites in a sense, profiting from the collective wisdom of active managers without paying for it. Though I'm not qualified to judge, it strikes me as dangerous to have multi-trillion dollar markets depend for price discovery on the active investment decisions of those managing a tiny fraction of the total assets.