Prediction Markets - PowerPoint PPT Presentation

1 / 83
About This Presentation

Prediction Markets


Prediction Markets Leighton Vaughan Williams Professor of Economics and Finance Nottingham Business School Nottingham Trent University – PowerPoint PPT presentation

Number of Views:465
Avg rating:3.0/5.0
Slides: 84
Provided by: gavin97


Transcript and Presenter's Notes

Title: Prediction Markets

Prediction Markets
Leighton Vaughan Williams Professor of Economics
and Finance Nottingham Business School Nottingham
Trent University
How Well Do Markets Aggregate Information?
  • How wise is the crowd?

Galtons Ox
  • In 1906, Sir Francis Galton (1822-1911), the
    English explorer, anthropologist and scientist,
    visited the West of England Fat Stock and Poultry
    Exhibition, where he came across a competition in
    which visitors could, for sixpence, guess the
    weight of an ox.
  • Those who guessed closest would receive prizes.
  • 800 people entered.

Galtons crowd
  • Many non-experts competed, like those clerks
    and others who have no knowledge of horses, but
    who bet on races, guided by newspapers, friends,
    and their own (Brief paper by Galton in
    Nature, March 1907).
  • Reference
  • F. Galton, Vox Populi, Nature, 75,
  • March 7, 1907.

Galtons findings
  • Galton added the contestants estimates and
    calculated the average of the estimates.
  • Using the mean, the crowd had guessed that the ox
    (slaughtered and dressed) would weigh 1,197
    pounds. In fact, the ox weighed 1,198 pounds.
  • The median estimate was 1,207 pounds, not as
    close but within 1 of the correct weight.

Treynors Jelly Beans Experiment
  • Jack Treynor, in a classic experiment, asked his
    class of 56 students to guess the number of jelly
    beans in a jar. The mean guess was 871.
  • The actual number was 850. Only one student
    guessed closer.
  • Reference Jack Treynor (1987), Market
    Efficiency and the Bean Jar Experiment,
    Financial Analysts Journal, 43, 50-53.
  • See also Kate Gordons seminal study of 200
    students estimating the weights of items. The
    group (average) result was 94.5 correct only 5
    students were better than this.
  • Kate H. Gordon (1921), Group Judgements in the
    Field of Lifted Weights, Psychological Review,
    28 (6), November, 398-424.

Webinar on Forecasting Excellence and Prediction
Markets, Sept. 15, 2007.
  • Joe Miles, a mathematician employed at
    eyepharma (a company offering services to the
    pharmaceutical industry) gave a presentation,
    with the following key points.
  • 1. He relayed the results of a MMs in a jar
    experiment he had conducted with a large group of
    conference delegates at a pharmaceutical
    forecasting conference earlier that year. The
    estimates ranged from 381 to over 40,000! The
    median estimate was 1,789. The actual number was
    1,747, just 2.4 off. The middle estimate was
    closer than any individual estimate.
  • 2. He relayed the results of an experiment
    conducted by eyetravel, a sister company, at a
    hotel industry conference. Delegates were asked
    to estimate the average price of a hotel room in
    Amsterdam that day. Estimates ranged by a factor
    of three, but the average estimate was just 0.5
    off (Mean estimate 117.8 Euro Actual price
    118.4 Euro.

What destroyed the space shuttle Challenger?
  • On January 28, 1986, the space shuttle Challenger
    lifted off from its launch pad at Cape Canaveral.
    Seventy-four seconds later, it blew up. Within
    minutes, investors started dumping the stocks of
    the four major contractors who had participated
    in the Challenger launch Rockwell International,
    which built the shuttle and its main engines
    Lockheed, which managed ground support Martin
    Marietta, which manufactured the ship's external
    fuel tank and Morton Thiokol, which built the
    solid-fuel booster rocket. Within minutes,
    trading in Thiokol was suspended and by the end
    of the day, Thiokol's stock was down nearly 12
    percent. By contrast, the stocks of the three
    other firms each fell a little but soon started
    to creep back up, and by the end of the day had
    fallen only around 3 percent. The market was
    right. Six months later and after an extensive
    investigation, Thiokol was held liable for the
    accident. The other companies were exonerated

How do you find a missing submarine?
  • On the afternoon of May 27, 1968, the submarine
    USS Scorpion was declared missing with all 99 men
    aboard. It was known that she must be lost at
    some point below the surface of the Atlantic
    Ocean within a circle 20 miles wide. This
    information was of some help, of course, but not
    enough to determine even five months later where
    she could actually be found.
  • The Navy had all but given up hope of finding the
    submarine when John Craven, who was their top
    deep-water scientist, came up with a plan which
    pre-dated the explosion of interest in prediction
    markets by decades. He simply turned to a group
    of submarine and salvage experts and asked them
    to bet on the probabilities of what could have
    happened. Taking an average of their responses,
    he was able to identify the location of the
    missing vessel to within a furlong (220 yards) of
    its actual location. The sub was found!

  • What are Prediction Markets?

Betting on the outcome
  • Betting markets aggregate all available
    information to produce best estimate, not least
    because those who know, and are best able to
    process the information, bet the most. Based on
    the Efficient Markets Hypothesis, the idea that
    markets accurately incorporate all relevant

Prediction markets v. Betting markets
  • The essential difference between prediction and
    betting markets is not an issue of structure.
  • Rather, prediction markets, as usually termed,
    are distinct from betting markets in the purpose
    to which they are put.
  • For example, when betting markets are used
    explicitly to forecast the outcome of any event,
    whether it is the World Cup or a rowing regatta,
    they are essentially acting as prediction
  • Even so, the term prediction markets often
    implies that the markets are being used to
    produce information externalities that can inform
    business and policy decisions.

The Hayek Question
  • How does one effectively aggregate disparate
    pieces of information that are spread among many
    different individuals, information that in its
    totality is needed to solve a problem?
  • Hayeks answer was that market prices are the
    means by which those disparate pieces of
    information are aggregated.
  • The mere fact that there is one price for any
    commodity ... brings about the solution which ...
    might have been arrived at by one single mind
    possessing all the information which is, in fact,
    dispersed among all the people involved in the
  • Source F.A. Hayek, The Use of Knowledge in
    Society, American Economic Review, 35, 4, Sept.
    1945 520.

(No Transcript)
Speed of the market in processing new information
  • Obama price spiked one day in August, 2008,
    despite the only obvious news being a relatively
    poor opinion poll.
  • Why?

Warp Speed Market Saddam capture or neutralize
  • Date 13 December, 2003
  • Market moves from about 20 to 100.
  • Next day News of Saddam capture announced by US.

(No Transcript)
Predicting the outcome of rowing regattas
  • Jed Christiansen (2007) reports on markets set up
    to predict the outcome of rowing regattas in the
    UK. Despite the small number of participants, and
    the absence of any incentives other than the
    challenge of getting it right, the predictions
    of the rowing events were highly accurate.
  • Christiansen puts the success of the experiment
    down to the effects of community and uniqueness,
    which encouraged motivated participation.
  • Source Christiansen, J.J. (2007), Prediction
    markets Practical experiments in small markets
    and behaviours observed, Journal of Prediction
    Markets, 1, 17-41.

Polls or markets?
  • Predicting the winner of an election!

Early prediction markets
  • The earliest data we have from prediction markets
    are those from organized markets for betting on
    the US Presidential election between 1868 and
  • Although there are reports that these markets
    date back to the election of George Washington,
    and even before, the market in 1868 seems to be
    the first we would call a prediction market in
    that its data was used to inform the public about
    the likelihood of a particular candidate winning
    and may have been used by financial asset
  • As an example, the New York Times reported that
    between 500,000 and 1 million was wagered on
    the Curb Exchange (the fore-runner to the AMEX)
    in one day on the 1916 election and that oil
    stocks were almost forgotten. The total amount
    wagered in these markets in 1916 was 165 million
    (at 2002 prices).
  • In this period between 1868 and 1940, the market
    failed to predict the winner on just one

  • Brecon and Radnor By-Election, 1985.
  • Mori v. Ladbrokes

  • MORI Labour to win by 18.
  • Liberal candidate 4 to 7
  • Labour candidate 5 to 4

  • The Liberal candidate.
  • Those who ignored MORI and backed the market

Bush v. Gore, 2000IG Index v. Rasmussen
Outcome forecasts
  • IG Index
  • 265-275 Bush
  • 265-275 Gore
  • Rasmussen Bush by 9

Opinion Polls v. markets
  • Opinion polls, like all market research, provide
    a valuable source of information, but they are
    ONLY ONE source of information.
  • Other information includes
  • 1. Local canvass returns
  • 2. On-the-ground inside information
  • 3. Forecasting models
  • 4. Opinions of professional pundits (experts)
  • 5. Focus groups
  • Betting markets aggregate all the available

Producing an optimal forecast
  • Because those who know the most, and are best
    able to process that information, tend to bet the
    most, this drives the market to produce an
    optimal forecast at any point in time.
  • Moreover, unlike polls, which are snapshots of
    opinion, betting markets are all about
    forecasting the eventual outcome.
  • Since the advent of zero-tax low-margin betting
    exchanges, the accuracy of these markets have
    improved yet further.

US Presidential Election 2004
  • INTRADE state-by-state predictions 50 out of 50.

British General Election, 2005
  • Predicted Labour majority to within a handful of

US Senate 2006
  • Intrade All correct

US Presidential Election 2008
  • Prediction markets
  • INTRADE state-by-state predictions 49 out of 50
    (called Missouri wrong).
  • BETFAIR state-by-state predictions 49 out of 50
    (called Indiana wrong).
  • Statistical Modelling Using Weighted Polling Data
  • FIVETHIRTYEIGHT predictions 48 out of 50 (called
    Missouri and North Carolina wrong).

  • Polls closed at 11.30 pm (UK time) in Indiana, a
    key state which McCain almost certainly needs to
    win to secure the Presidency.
  • McCain favourite to win Indiana on Betfair.
  • 11.45 pm Obama becomes favourite to win Indiana,
    attracting significant sums to win from traders.
  • By this time CNN was calling just 1 of precincts
    in Indiana.
  • So what caused the shift to Obama on Betfair?
  • In retrospect, it seems that professional traders
    had latched on to the detail in the few published
  • Importantly, this shows the power of prediction
    markets in assimilating and processing new
    information very rapidly.

Early Precinct Results
  • Stueben Kerry 34, Obama 42
  • De Kalb Kerry 31, Obama 38
  • Knox Kerry 36, Obama 54
  • Marshall Kerry 31, Obama 50
  • Only the most well-informed had accessed these
    results by 11.45, and knew what they meant, i.e.
    a big swing from Republican to Democrat since
    2004, but Betfair traders were among them.
    Minutes later, the swing was confirmed in Vigo
    County. By 12.20, Obama was shorter than 1 to 2
    on Betfair.

The election was called at 4am, but Betfair
watchers knew before midnight!
  • At 4am, California was declared, giving Obama the
    final few electoral votes required to win the
  • At 2.30am Ohio was called by most news networks.
  • Before midnight, the knowledge that Indiana was
    going to Obama, or at least that McCain would at
    best claim a small win there, was enough to
    indicate to Betfair watchers that the election
    was all but over. At 12.23 am, McCain was
    available at 25 to 1.
  • Meanwhile, Fox News declared that Indiana was
    over-polling for Obama because it shares a border
    with his home state of Illinois!
  • It was well past 3am when Fox News called the
    election for Obama.
  • Betfair 1, Fox News 0.

2010 UK General Election
  • It was the debates that lost it for the
  • Before the first debate, the markets all
    predicted a Conservative overall majority.
  • After the first debate, none of the markets ever
    predicted anything other than a hung parliament!
  • While the polls swung all over the place, the
    markets barely flickered after that first debate
    in predicting a hung parliament with the
    Conservatives the largest party with somewhere
    between 300 and 320 seats.

Could prediction markets have prevented 9/11?
  • The 9/11 Commission Report stated the problem
    like this "The biggest impediment to all-source
    analysis - to a greater likelihood of connecting
    the dots - is the human or systemic resistance to
    sharing information.
  • "What was missing in the intelligence community
    ... was any real means of aggregating not just
    information but also judgements. In other words,
    there was no mechanism to tap into the collective
    wisdom of National Security nerds, CIA spooks,
    and FBI agents. There was decentralization but
    not aggregation ... (James Surowiecki, The
    Wisdom of Crowds)
  • Can the market can help achieve this? Some people
    within the US Department of Defence had been
    working on just such an idea for several months
    when al Qaeda struck. Indeed, in May 2001 the
    Defense Advanced Research Projects Agency (DARPA)
    had issued a call for proposals under the heading
    of 'Electronics Market-Based Decision Support'
    (later 'Future Markets Applied to Prediction'

FutureMAP (cont.)
  • The remit prescribed for FutureMAP was to create
    market-based techniques for avoiding surprise and
    predicting future events. It was not long,
    however, before the US media and key members of
    the Congress began to train their guns on the
    idea of such a market. After all, it isn't
    difficult to portray the market as no more than a
    forum for eager traders to profit from death and
    destruction. The populist arguments won the day
    and DARPA was forced to cancel the project.
  • While most of the arguments against the market
    were emotional rather than intellectual, there
    was nevertheless some genuine intellectual
    concern as to how effective it would be likely to

Was Stiglitz right?
  • In particular, Prof. Joseph Stiglitz argued in an
    article published in the Los Angeles Times on 31
    July 2003 ('Terrorism There's No Futures in
    It'), that the market would be too "thin" (i.e.
    there would be too little money traded in the
    market) for it to be a useful tool for predicting
    events meaningfully. His argument was based on
    work he had previously published showing that
    markets can never be perfectly efficient when
    information is costly to obtain. The cost of
    obtaining and processing this information is, by
    implication, likely to act as a significant
    disincentive particularly in the context of a
    thin market (and hence low rewards).
  • But is it obviously the case that a properly
    constructed market, populated by suitably
    motivated (and perhaps screened) players can be
    viewed in this way?

Can prediction markets be used to study climate
  • 1) A properly constructed market might encourage
    climate change analysts to become more specific
    in their forecasts, and would encourage the
    development of new modelling techniques. 2) The
    markets could help to provide an assessment of
    the tangible impact upon climate change of
    various policies under consideration by
    governmental and international bodies.3) The
    market could potentially help to establish a
    price for carbon. 4) The markets could help to
    price in new information more quickly. 5) The
    market would help businesses and governments to
    hedge against both the dangers of climate change,
    and against costs of addressing it.
  • There could be a series of contracts and perhaps
    options on, for example, temperature, CO2
    emissions, precipitation, and tropical storms
    which expired at various intervals.

Can prediction markets help us make flight plans?
  • Volcanic ash cloud
  • BA Strike
  • Can we construct a prediction market which can
    amass the collective wisdom of the informed crowd
    to help us plan our future schedule?

Prediction Markets in Public Authorities
  • A notable feature of public policy in the UK over
    the past decade has been the imposition by
    central governments of performance targets as a
    means of evaluating the performance of local
    public organisations.
  • Targets cover a huge range of activities ranging
    from those specific to health or education to
    those relating to more general local authority
  • Targets are used as a means of evaluating
    performance, improving standards and allocating
    resources. The significance of achieving or not
    achieving particular targets can be very high for
    local politicians as well as senior managers in
    local authorities and health organisations in
    terms of both resources, public image. At the
    same time, it is extremely difficult for
    politicians and central managers to be aware of,
    let alone to process, the complex streams of
    information that are available

PMs in public authorities (cont.)
  • Within this context, prediction markets offer a
    potentially valuable tool that may be used to
    synthesize the specific knowledge of those
    directly involved with implementing policy at a
    lower level. The specific nature of targets
    relating to, e.g., waiting list times,
    educational outcomes, are both specific and
    quantifiable and, hence, ideal candidates for
    operating a trading market. Taking the example
    of health care targets, the numbers of people
    involved from nurses, doctors to administrators
    further suggest that the operation of markets in
    this context is feasible.
  • The value of the information provided by
    prediction markets will come primarily from the
    advance warning that politicians and managers
    will be given of weak performance in particular
    areas. This has the potential to improve
    resource allocation to make it more likely that
    key targets are met.

PMs for Public Policy Decisions
  • Example Should policy A or policy B be
    undertaken to reduce waiting lists?
  • Current waiting list for an appointment at the
    eye clinic 30 days.
  • Contract pays 1 for the length of the waiting
    list in days. And currently trades at 30 pounds.
  • Participants in the market can BUY the contract
    at 30 if they think the waiting list will
    increase and SELL if they think it will decrease.
  • E.g. If they SELL at 30 and the waiting list
    decreases to 25 days, they will 5 (30-25). But
    if the waiting list increases to 35 days, they
    lose 5.
  • By comparing the Waiting list with policy A
    contract with the Waiting list with policy B
    contract, the policy maker has gained information
    on what the market thinks about the relative
    impacts of introducing policy A and policy B on
    the length of the waiting list.
  • If a policy is not implemented, the contract is
    declared void.

Using the power of prediction markets for disease
  • http//
  • Reporting speed is one of the most import
    aspects of any surveillance program for seasonal
    influenza even two weeks advance notice can have
    dramatic results on the effectiveness of
  • Although there are many existing strategies for
    gathering opinions about the future trends of
    infectious diseases, the resulting data are often
    difficult to interpret using standard
    epidemiological methods. Prediction markets, on
    the other hand, are well known for their ability
    to quickly collect and summarize information.
  • The Iowa Electronic Health Markets is a research
    project at the University of Iowa exploring the
    use of prediction markets as a tool for disease
    surveillance. By combining the strengths of
    prediction markets with the knowledge of our
    trading community from around the world, our hope
    is that these markets will report future
    infectious disease activity quickly enough to be
    clinically useful.

Limitations of crowd wisdom
  • Can the crowd predict the lottery numbers?
  • If not, why not?
  • Because lottery numbers are drawn randomly, no
    model or individual or crowd or other means of
    aggregating information can predict them because
    random numbers are by definition unpredictable.
  • If the lottery numbers were, for whatever reason,
    not drawn randomly, however, we have a different

Is Manipulation Bad for Prediction Markets?
  • Robin Hanson and Ryan Oprea, of George Mason
    University and the University of California,
    Santa Cruz respectively, co-authored a paper
    title, 'A Manipulator Can Aid Prediction Market
    Accuracy. A perspective on its basic message is
    offered by Alex Tabarrok at Marginal Revolution.
    Tabarrok was considering the impact of the clear
    attempt by at least one determined trader to
    manipulate one of the US election betting markets
    in favour of Senator John McCain. In particular,
    the John McCain contract was bought in the
    markets systematically every morning by one
    US-based trader for sizeable sums. In
    consequence, it was possible to arbitrage between
    McCain (on Intrade) and Obama (on Betfair) for a
    few weeks in the run-up to Election 2008.
  • How much of a danger, Tabarrock asks, does this
    sort of activity pose for the whole concept of
    prediction markets? Not much, he argues, instead
    offering support for Hanson and Oprea's finding
    that manipulation can actually improve prediction
    markets, for the simple reason that manipulation
    offers informed investors a free lunch.

Manipulation (cont.)
  • "In a stock market", Tabarrok writes, "... when
    you buy (thinking the price will rise) someone
    else is selling (presumably thinking the price
    will fall) so if you do not have inside
    information you should not expect an above normal
    profit from your trade. But a manipulator sells
    and buys based on reasons other than expectations
    and so offers other investors a greater than
    normal return. The more manipulation, therefore,
    the greater the expected profit from betting
    according to rational expectations.
  • For this reason, investors should soon move to
    take advantage of any price discrepancies thus
    created within and between markets, as well as to
    take advantage of any perceived mispricing
    relative to fundamentals. Thus the expected value
    of the trading is a loss for the manipulator and
    a profit for the investors who exploit the
    mispricing. Moreover, the incentive the activity
    of the manipulator gives for others to become
    informed, and to trade on the basis of this
    information, is valuable in itself in improving
    the efficiency of the market.

Worth manipulating?
  • Tabarrok offers the additional observation that,
    considerations of predictive accuracy aside,
    there is one even more important lesson to be
    learned from the activities of the manipulators
    "...that prediction markets have truly arrived
    when people think they are worth manipulating".
  • But have they? What does the corporate sector

  • To take an example, a manufacturer of aero
    engines will seek good forecasts of future orders
    from plane manufacturers, which in turn will be
    contingent upon orders from airlines. Forecasts
    of future airline orders will be greatly assisted
    by the collation of a range of information from
    those involved in each of these sectors.
  • It is important that the questions posed are in a
    form which is unambiguous and which can
    ultimately be quantified. This requires an
    assessment of who should be involved in
    responding, and ensuring that each of these
    contributors has an equivalent understanding of
    the meaning of what is being asked, and that
    these answers can usefully be pooled. The set-up
    will vary depending on the diversity of
    contributors, both geographically and
    functionally. There is also the issue of
    incentives and the number of markets to run, as
    well as the length of these markets and how often
    new markets should be introduced.
  • But in principle markets should be able to help
    aggregate information.

But whats the evidence? Can prediction markets
actually help internal company forecasting?
  • There is in fact plenty of published research
    showing how internal prediction markets have
    helped improve the ability of commercial
    organisations to structure and implement internal
    prediction markets to assist in forecasting.
  • to predict key business variables
  • e.g. when will a product launch, what will be the
    unit sales?
  • broader-based prediction markets are a useful
    mechanism for predicting market-wide outcomes,
    e.g. box office receipts for a new film, success
    of a new video game, property prices.

Commercial examples
  • Eli Lilly ran an experiment in which managers
    traded through an internal market the future
    monthly sales figures for three drugs. The market
    brought together all the information, from
    toxicology reports to clinical results, and
    produced more accurate forecasts than the
    official forecasts.
  • Google have set up a market, in which any Google
    staff member could bet on the chances of an event
    coming true. The markets were used to forecast
    such things as product launch dates and new
    office openings. The results - based on the
    aggregated bets of thousands of Google staff
    members - were strong predictors of the actual

How do these markets operate?
  • Essentially, participants in the market exchange
    offers and counter-offers until they agree on a
    contract price.
  • Trades are executed when two prices match.
  • In describing how experimental internal-prediction
    market run by Eli Lilly (the pharmaceutical
    company) work, VP for Lilly Research Laboratories
    Alpheus Beingham noted
  • When we start trading in the drug and I try
    buying your stock cheaper and cheaper, it forces
    us to a way of agreeing that never really occurs
    in any other kind of conversation.

Corporate Applications of Prediction Markets
Special Issue of the JPM
  • The Journal of Prediction Markets (2009)
  • Guest editor Prof. Koleman Strumpf, member of
    the Editorial Board of the Journal of Prediction
    Markets, and Koch Professor of Economics at the
    University of Kansas School of Business.
  • Based on presentations at the Conference on
    Corporate Applications of Prediction/Information
    Markets, held at Kansas Citys Kauffman
    Foundation on 1 November, 2007.

What are Corporate Prediction Markets? Editorial
  • Prediction Markets use the knowledge of a pool
    of individuals to help forecast questions of
    importance to companies, such as whether a sales
    target will be reached or whether a project will
    be completed in a timely manner. A more recent
    development is the use of such markets to
    generate and evaluate new ideas, such as new
    products or cost saving procedures.
  • Since first being applied within the corporate
    sector over a decade ago, over a hundred
    companies have run internal markets. These range
    in size from some of the largest in the world to
    those with only a handful of employees, and cover
    a broad range of sectors, from those whose
    products are abstract ideas to others which
    manufacture very low-tech products.

Why Have Such a Broad Range of Firms Become
Interested in Prediction Markets?
  • The answer lies in a common problem facing
    firms, namely the isolation of executives from
    the views and insights of the companys
  • Such seclusion is no accident but instead
    reflects one of the reasons companies are
    structured as they are in the first place, i.e.
    to avoid information overload for busy
  • To reach this goal firms developed a hierarchy
    structure, and assigned to middle management the
    task of deciding how much and what information
    was transmitted from employees to higher-level
    decision-makers. But the system has its costs,
    as potentially useful information may be filtered
    out if it reflects poorly on those who control
    the information flow. At the same time,
    lower-level employees have little incentive to
    make reports which conflict with their managers.
    The net result is that executives may only
    receive one-sided information, and flawed
    decisions may result.

This is where Prediction Markets Come In!
  • Suppose the CEO must decide whether to continue
    funding a project, but is concerned that he has
    been receiving overly optimistic reports on its
    prospects from managers who will benefit from the
    project continuing.
  • A market on the projects prospects would allow
    front-line employees to convey more realistic
    information, and they could do so without fear of
    reprisal if the trading is anonymous.
  • Prediction markets may also function better than
    other approaches currently in use. For example,
    group meetings are less likely to have frank
    discussions while suggestions boxes do not scale
    well - prediction markets tend to perform better
    when there are more participants.
  • And while most workers may dread the thought of
    meetings, markets are often considered a fun
    activity and often do not require much in the way
    of incentives to generate active employee

A Case Study of GE (JPM, 2009), by Brian Spears
(Risk Manager, GE-Hitachi Nuclear Energy) et al.
  • 1. Internal markets used to aggregate opinions
    are consistent with opinions collected via web
    surveys (Chan et al, 2002). Markets may in fact
    improve upon traditional survey methods by
    encouraging greater honesty from the
    participants, providing participants with
    valuable feedback from other participants, and
    offering participants the joy of competitive
  • 2. GEs markets are designed to help answer
    business questions such as What new technology
    ideas should we be investing in? What new
    products should we be developing?
  • 3. Market participants can submit their own ideas
    for entry into the market, and they can buy and
    sell shares of any idea in the market.

GE Case Study (cont.)
  • GEs interest in idea markets stem from our
    belief that innovative new product and service
    ideas can come from anywhere within an
  • Similar to most companies, GE uses a variety of
    methods to generate and down-select new ideas.
    Traditional means include suggestion boxes and
    brainstorming sessions.
  • However, suggestion boxes often go unused because
    contributors receive little or no feedback about
    their idea or visibility into others ideas.
  • Brainstorming sessions are often infeasible for
    soliciting ideas from large, globally distributed
    teams with potentially thousands of contributors.
  • Hence the idea of a prediction market, which they
    call an Imagination Market was born.

Conclusions of Case Study
  • Overall, the GE Energy business was extremely
    pleased with the results of the Imagination
  • Funding was immediately provided to kick-start
    the two ideas tied for the top, and the business
    has decided to file patents for several others.
  • GE Energy plans to continue use of markets in the
  • The volume and quality of ideas compared
    favourably to brainstorming sessions, on-line
    suggestion boxes, and on-line discussion forums.
  • NB The Spears paper provides a wealth of
    information and detail about the GE markets,
    including a very detailed description of the
    mechanics of the markets, such as the incentives
    given to traders and to creators of new ideas.

Improving Forecasting Accuracy in Corporate
Prediction Markets A Case Study in the Austrian
Mobile Communication Industry JPM, 3, 2009, by
Martin Waitz and Andreas Mild
  • Corporate prediction markets forecast business
    issues like market shares, sales volumes or the
    success rates of new product developments.
  • The improvement of its accuracy is a major topic
    in prediction market research ...
  • We propose a method that aggregates the data
    provided by such a prediction market in a
    different way by only accounting for the most
    knowledgeable market participants.
  • We demonstrate its predictive ability with a real
    world experiment.

Why companies are well positioned to utilize the
information generated from PMs
  • 1. Company divisions often serve as standalone
    silos, and markets can be a means of integrating
    the pockets of information contained in each.
  • 2. Executives may be interested not just in
    market aggregates, such as prices, but also the
    trades of particular groups of employees. For
    example, one could examine whether members of
    certain divisions are less prone to making biased
  • 3. Companies need real-time information about the
    many uncertain events surrounding their
  • 4. Firms can internalize the informational
    benefits of the market. A company can profit from
    the information generated from prices, since the
    market can be kept private and outside of the
    purview of competitors.
  • The last point is particularly important. Since
    the benefits of the markets largely accrue to the
    company, we might expect many prediction market
    innovations to first arise in a corporate

Key challenges facing PMs
  • 1. Operators must overcome investor reluctance to
    a project with upfront costs and possibly delayed
  • 2. There are impacts on employees, both
    detrimental (markets may distract staff away from
    their main responsibilities) and beneficial
    (there is often a gain in morale, as workers feel
    empowered because their market-mediated
    suggestions are impacting corporate decisions.
  • 3. The markets may overwhelm executives with too
    much information.
  • 4. Market organizers must allay concerns of
    middle management and those whose current role in
    the company is threatened by the market.
  • 5. There may be systematic biases in some markets.

Optimism bias
  • Optimism bias is the systematic tendency for
    people to be over-optimistic about the outcome of
    planned actions. This includes over-estimating
    the likelihood of positive events and
    under-estimating the likelihood of negative
  • David Armor and Shelley Taylor highlight a number
    of examples of what they consider to be optimism
    bias in an interesting paper called 'When
    Predictions Fail The Dilemma of Unrealistic
    Optimism', published in 2002.
  • Examples include students' estimates of the
    likely starting salary of their first job in the
    graduate market and newlyweds' thoughts on how
    long their marriage will last. It is interesting,
    therefore, that evidence of the existence of this
    very same bias has been identified in 'internal'
    company prediction markets, notably in a 2008
    paper co-authored by Bo Cowgill, of Google,
    Justin Wolfers of the Wharton School and Eric
    Zitzewitz, based at Dartmouth College.

Optimism bias (cont.)
  • Cowgill, Wolfers and Zitzewitz examine the
    results generated by what they call the Google
    corporate prediction market experiment. The
    primary goal of these markets is, as they put it,
    to generate predictions that efficiently
    aggregate many employees' information and augment
    existing forecasting methods.
  • In support of previous investigations into the
    value of internal prediction markets, they were
    able to confirm that prices in the Google markets
    closely approximated event probabilities, i.e.
    that the markets were reasonably efficient. Even
    so, they were not perfect, and one notable reason
    was an apparent 'optimism bias' which, according
    to their findings, "was more pronounced for
    subjects under the control of Google employees,
    such as whether a project would be completed on
    time or whether a particular office would be

Optimism bias (cont.)
  • Optimism bias was also found to be more evident
    in new employees and in the immediate few days
    following a good news day for the Google stock
    price. Still, what is a cost in terms of
    unadjusted predictive efficiency may be a benefit
    in terms of motivation and entrepreneurial zeal,
    a feedback mechanism the value of which it is
    perhaps easy to under-estimate.
  • In any case, if we are able to identify and
    measure the source and extent of the bias, it
    should be possible to adjust and compensate for
    this particular inefficiency in generating the
    objective forecasts.

The Favourite-Longshot Bias
  • Let the probability of an event occurring be 20.
  • Standard approach Probability 0.2
  • Bookmakers approach Odds 4 to 1. This means
    you win 4 (net) from the bookmaker if your bet
    wins for every 1 staked (risked) with the
  • Which yields the better expected return, a stake
    of 10 on a horse with odds of 2 to 1 or a stake
    of 10 on a horse with odds of 20 to 1?
  • i.e. if Mr. A and Mr. B both start with 1,000.
    Now Mr. A places a level 10 stake on 100 horses
    quoted at 2 to 1 and Mr. B places a level 10
    stake on 100 horses quoted at 5 to 1. Who is
    likely to end up richer?

Odds Backing Ladbrokes Pocket Companion, Flat
Edition, 1990, pp. 242-243
  • Not one out of 35 favourites sent off at 1/8 or
    shorter (as short as 1/25) lost between 1985 and
    1989. This means a return of between 4 and 12.5
    in a couple of minutes, which is an astronomical
    rate of interest... The point being made is that
    broadly speaking the shorter the odds, the better
    the return. More broadly, the group of white
    hot favourites (odds between 1 to 5 and 1 to 25)
    won 88 out of 96 races for a 6.5 profit. The
    following table looks at other odds groupings.
  • Odds Wins Runs Profit
  • 1/5-1/2 249 344 1.80 0.52
  • 4/7-5/4 881 1780 -82.60 -4.64
  • 6/4 -3/1 2187 7774 -629 -8.09
  • 7/2-6/1 3464 21681 -2237 -10.32
  • 8/1-20/1 2566 53741 -19823 -36.89
  • 25/1 -100/1 441 43426 -29424 -67.76

An Election Super-Bias?
  • The 2004 US Presidential state-by-state markets
    gave the equivalent of 50 successive winning
    favourites at the racetrack.
  • In 2008, the Betfair favourites won 49/50 states.
  • The Intrade favourites won 49/50.

  • Do biases differ between different prediction
    market formats?
  • Can we compensate for the biases to yield more
    accurate forecasts?
  • Do some formats yield more volatile forecasts
    than others?

Some further examples of PMs
  • Best Buy, the electronics retailer. Experimented
    with prediction markets on everything from demand
    for digital set-top boxes to store opening-dates.
  • E.g. in Autumn 2006, the price in one of their
    PMs on whether a new store in Shanghai would open
    on time several weeks ahead dropped sharply from
    80 a share to about 45. Players made yes-no
    bets, and the virtual dollar drop reflected
    increasing doubt that the store would open on
    time. The store opened a month late.
  • Jeffrey Severts, a VP who oversees PMs at Best
    Buy, quoted in NY Times (April 9, 2008 Betting
    to Improve the Odds)
  • The potential is that prediction markets may be
    the thing that enables a big company to act more
    like a small, nimble company again.
  • It helps on two fronts, the speed and accuracy
    of information, so that management can move
    faster to deal with problems or exploit

Best Buy (cont.)
  • Severts invited several hundred employees to
    submit an estimate for sales for a one-month
    period. To help them calibrate their estimate, he
    provided monthly data from the past twelve
    months. 192 employees responded, including those
    on the store floor. The estimates were given
    equal weight and averaged. He found that the
    employees collective wisdom had an error of only
    0.5 compared to an error of 5 by the
    traditional forecasting method.
  • Severts went on to experiment with total sales
    over the 14 week holiday period. He provided last
    years sales figure from the holiday period and
    revenue growth for the first three months of the
    current fiscal year compared to previous years.
    The original 350 respondents predicted sales
    during the fourteen-week holiday period that was
    99.9 accurate. The merchants themselves who were
    traditionally responsible for forecasting were
    93 accurate.
  • (Hamel, 2007, The Future of Management, Boston
    Harvard Business School Press).

and Vladimir Vladimirov (JPM, 2009)
  • This paper presents the outcome of a study
    conducted at the Royal Devon and Exeter Hospital
    in which a prediction market was established in
    order to forecast demand for services.
  • The study was conducted over a period of one
    week, and involved 65 participants. Each was
    asked to provide an estimate for demand for
    services at the Royal Devon and Exeter Hospital.
    In each survey, each employee was asked to
    estimate the number of patients that would be
    admitted to each directorate, which meant that
    employees within each directorate were estimating
    the number of patients admitted to directorates
    other than their own.

  • The overall results confirmed the effectiveness
    of prediction markets.
  • The prediction for admittances was 1157.51 while
    the actual number of admittances was 1154, which
    is an error of only 0.3.
  • Market participants were almost exactly right in
    the Medicine directorate, predicting 353.38,
    while the actual was 353.
  • Specialist Surgerys prediction was almost as
    accurate with an actual number of admittances of
    106 and an estimate of 107.75.
  • However, the prediction market was less
    successful in predicting demand for services for
    each department, which the paper attributes to
    the small sample size and lack of diversity of

Further examples
  • In 2007, a group in the purchasing unit at
    Hewlett Packard began prediction markets on the
    price of computer memory chips three and six
    months ahead.
  • Bernardo A. Huberman, director of the social
    computing lab at Hewlett-Packard The prediction
    markets were up to 70 more accurate than the
    companys traditional forecasting model ... The
    more accurate predictions can be used to finesse
    purchasing, marketing and product pricing

The Classic Study
  • Information Aggregation Mechanisms Concept,
    Design and Implementation for a Sales Forecasting
    Problem, by Charles Plott (CalTech) and Kay-Yut
    Chen (Hewlett-Packard Laboratories), 2002.
  • Many business examples share the following
    characteristic small bits and pieces of relevant
    information exists in the opinions and intuition
    of individuals who are close to an activity. Some
    examples are supply chain management issues,
    demand forecasting, new product introduction, and
    supply uncertainties. In many instances, no
    systematic methods of collecting such information
    exist. In these cases very little is known by any
    single individual but the aggregation of the bits
    and pieces of information might be considerable.
    For instance, it is extremely difficult to
    combine subjective information such as the
    knowledge of a competitors move with objective
    information such as historical data. In a perfect
    world, with unlimited time and resources, a user
    of such information could personally interview
    everyone that might have a
  • relevant insight but such luxury does not exist.

Chen and Plott (cont.)
  • Gathering the bits and pieces by traditional
    means, such as business meetings, is highly
    inefficient because of a host of practical
    problems related to location, incentives, the
    insignificant amounts of information in any one
    place, and even the absence of a methodology for
    gathering it. Furthermore, business practices
    such a quotas and budget settings create
    incentives for individuals not to reveal their
    information. The principles of economics together
    with new technologies that exist for creating
    markets and related mechanisms suggest that in
    might be possible to develop a new approach that
    avoids many of the practical problems.

Some details of the experiment
  • The experiments were conducted with three
    different HP divisions. Typically, around 20-30
    people signed up for the experiments. Trading was
    done through at a web server located at Caltech.
    The subjects were geographically dispersed in
  • Typically, the prediction was for monthly sales
    for a month three months in the future. The
    market mechanism employed to support the markets
    was the web based markets of the Marketscape
    software, which was
  • developed at the Laboratory of Economics and
    Political Science at Caltech. All the markets for
    an event were organized on a single web page for
    easy access.
  • A participant could enter a buy offer, a sell
    offer or acceptance of an offer through the web
    form on the page. Orders were compared to the
    other side immediately. The best offers were
    listed on the main market web page. The whole
    book of offers was available for each market at
    the click of a button.

(No Transcript)
  • Market predictions based on IAM (Information
    Aggregation Mechanism) prices outperformed
    official HP forecasts.
  • In events for which official forecasts were
    available the IAM predictions were closer to the
    actual outcome than the official forecast 75 of
    the time. The absolute errors of the official
    forecasts were also significantly higher than
    that of the IAM predictions.

Other general applications of PMs
  • Examples of the applications of prediction
    markets range from traditional finance
    forecasting such as sales and costs, to product
    development support such as forecasting on-time
    project delivery or the likelihood of regulatory
    approval for new drugs, to innovative decision
    support such as evaluating the impact of
    switching advertising agencies or forecasting the
    market receptivity of new software releases
  • Etc
  • Etc
  • Etc.

Its not just about forecasting!
  • A prediction-market pilot at Microsoft in 2005
    was designed to forecast the probability of
    on-time release for several products.
  • To managements surprise, the stock price
    representing on-time release dropped to zero,
    despite the staffs prior assurance that on-time
    release was likely.
  • The ensuing conversation uncovered the true
    beliefs of the programmers, a result perhaps even
    more valuable than knowing whether the release
    would be missed.
  • Source Todd Proebsting, of Microsoft Research,
    in his presentation Tee Time with Admiral
    Poindexter, delivered at the DIMACS Workshop on
    Market as Predictive Devices (Information
    Markets), February, 2005, Rutgers University.

  • Prediction markets offer major unexploited
    opportunities to aggregate information in a rapid
    and efficient manner. There are significant
    examples of success where they have been tried,
    although in some cases they have been tried and
    then discarded.
  • In an interview with DIRECTOR magazine, I said
  • The real problem where these markets have not
    worked as well as management expected is that the
    companies have simply bought the technology and
    more or less expected it to take care of itself.
    I dont want to stretch the point but this can be
    compared to buying a car and expecting it to
    drive itself. Of course youd be disappointed
    with performance.
  • And yes cars have design and performance
    glitches, as do PMs.
  • But cars can be useful and wonderful things.
  • The same, may I say it, goes for Prediction

Write a Comment
User Comments (0)