Prediction markets

Outside the political sphere, there are a growing number of web-based prediction markets, often run by companies that provide a range of trading and gambling services. Some prominent examples include Tradesports.com (also known as InTrade.com), Betfair.com, and pseudo-markets (in which participants trade virtual currency) such as Newsfutures.com and Ideosphere.com.

Some prediction markets focus on economic statistics. For instance, Goldman Sachs, Deutsche Bank, and the Chicago Mercantile Exchange have launched markets on the likely outcome of future readings of economic statistics, including employment, retail sales, industrial production, and business confidence.

How accurate are prediction market forecasts?

Arguably the most important issue with these markets is their performance as predictive tools. In the political domain, the Iowa Electronic Markets has both yielded very accurate predictions for United States elections, and also outperformed the major pollsters.

In a comparison of election-eve forecasts, the Iowa markets predicted candidates’ vote shares with an average absolute error of 1½ percentage points, while the average forecast error for Gallup polls was 2.1 percentage points.

In Australia, election-eve betting markets correctly predicted that the Coalition would win the 2001 and 2004 federal elections. By contrast, a Coalition win was predicted by the election-eve polls of only two out of three major pollsters in 2001, and only two out of four major pollsters in 2004.

Perhaps more interesting in terms of how well prediction markets can aggregate information is the performance of markets at the level of the individual district. Typically districts are sufficiently small that there is little interest (or funding) for local polling, yet when Australian bookmakers started betting on district-level races, we found that they were extremely accurate.

In 2001, the Centrebet favourite won in 43 out of 47 marginal seats. In 2004, the Centrebet favourite won in 24 out of 32 marginals.

That said, comparing the performance of markets with a mechanistic application of poll-based forecasting may not provide a particularly compelling comparison. A more relevant starting point might be to compare the predictions of markets with those of independent analysts.

For example, a survey of ten experts published on the Sunday before the 2004 federal election found that three thought Mark Latham would win, while seven thought John Howard would win, but with a smaller majority than in 2001. None forecast the true result - a Howard victory with an increased majority. By contrast, the betting market was more confident of a Howard victory in 2004 than in 2001.

In a corporate context, the Hollywood Stock Exchange predicts opening weekend box-office success, and these predictions have been quite accurate. Further, this market has been about as accurate at forecasting Oscar winners as an expert panel.

Some firms have also begun to experiment with internal prediction markets. An internal market at Hewlett-Packard produced more accurate forecasts of printer sales than the firm’s internal processes.

Gerhard Ortner described an experiment at Siemens in which an internal market predicted that the firm would definitely fail to deliver on a software project on time, even when traditional planning tools suggested that the deadline could be met.