# Prediction Markets - PowerPoint PPT Presentation

1 / 83
Title:

## Prediction Markets

Description:

### Prediction Markets Leighton Vaughan Williams Professor of Economics and Finance Nottingham Business School Nottingham Trent University Leighton.Vaughan-Williams_at_ntu.ac.uk – PowerPoint PPT presentation

Number of Views:465
Avg rating:3.0/5.0
Slides: 84
Provided by: gavin97
Category:
Tags:
Transcript and Presenter's Notes

Title: Prediction Markets

1
Prediction Markets
Leighton Vaughan Williams Professor of Economics
and Finance Nottingham Business School Nottingham
Trent University Leighton.Vaughan-Williams_at_ntu.ac.
uk
2
How Well Do Markets Aggregate Information?
• How wise is the crowd?

3
Galtons Ox
• In 1906, Sir Francis Galton (1822-1911), the
English explorer, anthropologist and scientist,
visited the West of England Fat Stock and Poultry
Exhibition, where he came across a competition in
which visitors could, for sixpence, guess the
weight of an ox.
• Those who guessed closest would receive prizes.
• 800 people entered.

4
Galtons crowd
• Many non-experts competed, like those clerks
and others who have no knowledge of horses, but
who bet on races, guided by newspapers, friends,
and their own (Brief paper by Galton in
Nature, March 1907).
• Reference
• F. Galton, Vox Populi, Nature, 75,
• March 7, 1907.

5
Galtons findings
• Galton added the contestants estimates and
calculated the average of the estimates.
• Using the mean, the crowd had guessed that the ox
(slaughtered and dressed) would weigh 1,197
pounds. In fact, the ox weighed 1,198 pounds.
• The median estimate was 1,207 pounds, not as
close but within 1 of the correct weight.

6
Treynors Jelly Beans Experiment
• Jack Treynor, in a classic experiment, asked his
class of 56 students to guess the number of jelly
beans in a jar. The mean guess was 871.
• The actual number was 850. Only one student
guessed closer.
• Reference Jack Treynor (1987), Market
Efficiency and the Bean Jar Experiment,
Financial Analysts Journal, 43, 50-53.
• See also Kate Gordons seminal study of 200
students estimating the weights of items. The
group (average) result was 94.5 correct only 5
students were better than this.
• Kate H. Gordon (1921), Group Judgements in the
Field of Lifted Weights, Psychological Review,
28 (6), November, 398-424.

7
Webinar on Forecasting Excellence and Prediction
Markets, Sept. 15, 2007.
• Joe Miles, a mathematician employed at
eyepharma (a company offering services to the
pharmaceutical industry) gave a presentation,
with the following key points.
• 1. He relayed the results of a MMs in a jar
experiment he had conducted with a large group of
conference delegates at a pharmaceutical
forecasting conference earlier that year. The
estimates ranged from 381 to over 40,000! The
median estimate was 1,789. The actual number was
1,747, just 2.4 off. The middle estimate was
closer than any individual estimate.
• 2. He relayed the results of an experiment
conducted by eyetravel, a sister company, at a
hotel industry conference. Delegates were asked
to estimate the average price of a hotel room in
Amsterdam that day. Estimates ranged by a factor
of three, but the average estimate was just 0.5
off (Mean estimate 117.8 Euro Actual price
118.4 Euro.

8
What destroyed the space shuttle Challenger?
• On January 28, 1986, the space shuttle Challenger
lifted off from its launch pad at Cape Canaveral.
Seventy-four seconds later, it blew up. Within
minutes, investors started dumping the stocks of
the four major contractors who had participated
in the Challenger launch Rockwell International,
which built the shuttle and its main engines
Lockheed, which managed ground support Martin
Marietta, which manufactured the ship's external
fuel tank and Morton Thiokol, which built the
solid-fuel booster rocket. Within minutes,
trading in Thiokol was suspended and by the end
of the day, Thiokol's stock was down nearly 12
percent. By contrast, the stocks of the three
other firms each fell a little but soon started
to creep back up, and by the end of the day had
fallen only around 3 percent. The market was
right. Six months later and after an extensive
investigation, Thiokol was held liable for the
accident. The other companies were exonerated

9
How do you find a missing submarine?
• On the afternoon of May 27, 1968, the submarine
USS Scorpion was declared missing with all 99 men
aboard. It was known that she must be lost at
some point below the surface of the Atlantic
Ocean within a circle 20 miles wide. This
information was of some help, of course, but not
enough to determine even five months later where
she could actually be found.
• The Navy had all but given up hope of finding the
submarine when John Craven, who was their top
deep-water scientist, came up with a plan which
pre-dated the explosion of interest in prediction
markets by decades. He simply turned to a group
of submarine and salvage experts and asked them
to bet on the probabilities of what could have
happened. Taking an average of their responses,
he was able to identify the location of the
missing vessel to within a furlong (220 yards) of
its actual location. The sub was found!

10
• What are Prediction Markets?

11
Betting on the outcome
• Betting markets aggregate all available
information to produce best estimate, not least
because those who know, and are best able to
process the information, bet the most. Based on
the Efficient Markets Hypothesis, the idea that
markets accurately incorporate all relevant
information.

12
Prediction markets v. Betting markets
• The essential difference between prediction and
betting markets is not an issue of structure.
• Rather, prediction markets, as usually termed,
are distinct from betting markets in the purpose
to which they are put.
• For example, when betting markets are used
explicitly to forecast the outcome of any event,
whether it is the World Cup or a rowing regatta,
they are essentially acting as prediction
markets.
• Even so, the term prediction markets often
implies that the markets are being used to
produce information externalities that can inform
business and policy decisions.

13
The Hayek Question
• How does one effectively aggregate disparate
pieces of information that are spread among many
different individuals, information that in its
totality is needed to solve a problem?
• Hayeks answer was that market prices are the
means by which those disparate pieces of
information are aggregated.
• The mere fact that there is one price for any
commodity ... brings about the solution which ...
might have been arrived at by one single mind
possessing all the information which is, in fact,
dispersed among all the people involved in the
process.
• Source F.A. Hayek, The Use of Knowledge in
Society, American Economic Review, 35, 4, Sept.
1945 520.

14
(No Transcript)
15
Speed of the market in processing new information
• Obama price spiked one day in August, 2008,
despite the only obvious news being a relatively
poor opinion poll.
• Why?

16
Warp Speed Market Saddam capture or neutralize
• Date 13 December, 2003
• Market moves from about 20 to 100.
• Next day News of Saddam capture announced by US.

17
(No Transcript)
18
Predicting the outcome of rowing regattas
• Jed Christiansen (2007) reports on markets set up
to predict the outcome of rowing regattas in the
UK. Despite the small number of participants, and
the absence of any incentives other than the
challenge of getting it right, the predictions
of the rowing events were highly accurate.
• Christiansen puts the success of the experiment
down to the effects of community and uniqueness,
which encouraged motivated participation.
• Source Christiansen, J.J. (2007), Prediction
markets Practical experiments in small markets
and behaviours observed, Journal of Prediction
Markets, 1, 17-41.

19
Polls or markets?
• Predicting the winner of an election!

20
Early prediction markets
• The earliest data we have from prediction markets
are those from organized markets for betting on
the US Presidential election between 1868 and
1940.
• Although there are reports that these markets
date back to the election of George Washington,
and even before, the market in 1868 seems to be
the first we would call a prediction market in
that its data was used to inform the public about
the likelihood of a particular candidate winning
and may have been used by financial asset
• As an example, the New York Times reported that
between 500,000 and 1 million was wagered on
the Curb Exchange (the fore-runner to the AMEX)
in one day on the 1916 election and that oil
stocks were almost forgotten. The total amount
wagered in these markets in 1916 was 165 million
(at 2002 prices).
• In this period between 1868 and 1940, the market
failed to predict the winner on just one
occasion.

21
AN EARLY BRITISH PREDICTION MARKET
• Brecon and Radnor By-Election, 1985.
• Mori v. Ladbrokes

22
ELECTION EVE
• MORI Labour to win by 18.
• Liberal candidate 4 to 7
• Labour candidate 5 to 4

23
WINNERS
• The Liberal candidate.
• Those who ignored MORI and backed the market
favourite.

24
Bush v. Gore, 2000IG Index v. Rasmussen
25
Outcome forecasts
• IG Index
• 265-275 Bush
• 265-275 Gore
• Rasmussen Bush by 9

26
Opinion Polls v. markets
• Opinion polls, like all market research, provide
a valuable source of information, but they are
ONLY ONE source of information.
• Other information includes
• 1. Local canvass returns
• 2. On-the-ground inside information
• 3. Forecasting models
• 4. Opinions of professional pundits (experts)
• 5. Focus groups
• Betting markets aggregate all the available
information

27
Producing an optimal forecast
• Because those who know the most, and are best
able to process that information, tend to bet the
most, this drives the market to produce an
optimal forecast at any point in time.
• Moreover, unlike polls, which are snapshots of
opinion, betting markets are all about
forecasting the eventual outcome.
• Since the advent of zero-tax low-margin betting
exchanges, the accuracy of these markets have
improved yet further.

28
US Presidential Election 2004
• INTRADE state-by-state predictions 50 out of 50.

29
British General Election, 2005
• Predicted Labour majority to within a handful of
seats.

30
US Senate 2006
• Intrade All correct

31
US Presidential Election 2008
• Prediction markets
• INTRADE state-by-state predictions 49 out of 50
(called Missouri wrong).
• BETFAIR state-by-state predictions 49 out of 50
(called Indiana wrong).
• Statistical Modelling Using Weighted Polling Data
• FIVETHIRTYEIGHT predictions 48 out of 50 (called
Missouri and North Carolina wrong).

32
Indiana
• Polls closed at 11.30 pm (UK time) in Indiana, a
key state which McCain almost certainly needs to
win to secure the Presidency.
• McCain favourite to win Indiana on Betfair.
• 11.45 pm Obama becomes favourite to win Indiana,
attracting significant sums to win from traders.
• By this time CNN was calling just 1 of precincts
in Indiana.
• So what caused the shift to Obama on Betfair?
• In retrospect, it seems that professional traders
had latched on to the detail in the few published
results.
• Importantly, this shows the power of prediction
markets in assimilating and processing new
information very rapidly.

33
Early Precinct Results
• Stueben Kerry 34, Obama 42
• De Kalb Kerry 31, Obama 38
• Knox Kerry 36, Obama 54
• Marshall Kerry 31, Obama 50
• Only the most well-informed had accessed these
results by 11.45, and knew what they meant, i.e.
a big swing from Republican to Democrat since
2004, but Betfair traders were among them.
Minutes later, the swing was confirmed in Vigo
County. By 12.20, Obama was shorter than 1 to 2
on Betfair.

34
The election was called at 4am, but Betfair
watchers knew before midnight!
• At 4am, California was declared, giving Obama the
final few electoral votes required to win the
Presidency.
• At 2.30am Ohio was called by most news networks.
• Before midnight, the knowledge that Indiana was
going to Obama, or at least that McCain would at
best claim a small win there, was enough to
indicate to Betfair watchers that the election
was all but over. At 12.23 am, McCain was
available at 25 to 1.
• Meanwhile, Fox News declared that Indiana was
over-polling for Obama because it shares a border
with his home state of Illinois!
• It was well past 3am when Fox News called the
election for Obama.
• Betfair 1, Fox News 0.

35
2010 UK General Election
• It was the debates that lost it for the
Conservatives!!!
• Before the first debate, the markets all
predicted a Conservative overall majority.
• After the first debate, none of the markets ever
predicted anything other than a hung parliament!
• While the polls swung all over the place, the
markets barely flickered after that first debate
in predicting a hung parliament with the
Conservatives the largest party with somewhere
between 300 and 320 seats.

36
Could prediction markets have prevented 9/11?
• The 9/11 Commission Report stated the problem
like this "The biggest impediment to all-source
analysis - to a greater likelihood of connecting
the dots - is the human or systemic resistance to
sharing information.
• "What was missing in the intelligence community
... was any real means of aggregating not just
information but also judgements. In other words,
there was no mechanism to tap into the collective
wisdom of National Security nerds, CIA spooks,
and FBI agents. There was decentralization but
not aggregation ... (James Surowiecki, The
Wisdom of Crowds)
• Can the market can help achieve this? Some people
within the US Department of Defence had been
working on just such an idea for several months
when al Qaeda struck. Indeed, in May 2001 the
Defense Advanced Research Projects Agency (DARPA)
had issued a call for proposals under the heading
of 'Electronics Market-Based Decision Support'
(later 'Future Markets Applied to Prediction'
(FutureMAP).

37
FutureMAP (cont.)
• The remit prescribed for FutureMAP was to create
market-based techniques for avoiding surprise and
predicting future events. It was not long,
however, before the US media and key members of
the Congress began to train their guns on the
idea of such a market. After all, it isn't
difficult to portray the market as no more than a
forum for eager traders to profit from death and
destruction. The populist arguments won the day
and DARPA was forced to cancel the project.
• While most of the arguments against the market
were emotional rather than intellectual, there
was nevertheless some genuine intellectual
concern as to how effective it would be likely to
be.

38
Was Stiglitz right?
• In particular, Prof. Joseph Stiglitz argued in an
article published in the Los Angeles Times on 31
July 2003 ('Terrorism There's No Futures in
It'), that the market would be too "thin" (i.e.
there would be too little money traded in the
market) for it to be a useful tool for predicting
events meaningfully. His argument was based on
work he had previously published showing that
markets can never be perfectly efficient when
information is costly to obtain. The cost of
obtaining and processing this information is, by
implication, likely to act as a significant
disincentive particularly in the context of a
thin market (and hence low rewards).
• But is it obviously the case that a properly
constructed market, populated by suitably
motivated (and perhaps screened) players can be
viewed in this way?

39
Can prediction markets be used to study climate
change?
• 1) A properly constructed market might encourage
climate change analysts to become more specific
in their forecasts, and would encourage the
development of new modelling techniques. 2) The
markets could help to provide an assessment of
the tangible impact upon climate change of
various policies under consideration by
governmental and international bodies.3) The
market could potentially help to establish a
price for carbon. 4) The markets could help to
price in new information more quickly. 5) The
market would help businesses and governments to
hedge against both the dangers of climate change,
and against costs of addressing it.
• There could be a series of contracts and perhaps
options on, for example, temperature, CO2
emissions, precipitation, and tropical storms
which expired at various intervals.

40
Can prediction markets help us make flight plans?
• Volcanic ash cloud
• BA Strike
• Can we construct a prediction market which can
amass the collective wisdom of the informed crowd
to help us plan our future schedule?

41
Prediction Markets in Public Authorities
• A notable feature of public policy in the UK over
the past decade has been the imposition by
central governments of performance targets as a
means of evaluating the performance of local
public organisations.
• Targets cover a huge range of activities ranging
from those specific to health or education to
those relating to more general local authority
performance.
• Targets are used as a means of evaluating
performance, improving standards and allocating
resources. The significance of achieving or not
achieving particular targets can be very high for
local politicians as well as senior managers in
local authorities and health organisations in
terms of both resources, public image. At the
same time, it is extremely difficult for
politicians and central managers to be aware of,
let alone to process, the complex streams of
information that are available

42
PMs in public authorities (cont.)
• Within this context, prediction markets offer a
potentially valuable tool that may be used to
synthesize the specific knowledge of those
directly involved with implementing policy at a
lower level. The specific nature of targets
relating to, e.g., waiting list times,
educational outcomes, are both specific and
quantifiable and, hence, ideal candidates for
operating a trading market. Taking the example
of health care targets, the numbers of people
involved from nurses, doctors to administrators
further suggest that the operation of markets in
this context is feasible.
• The value of the information provided by
prediction markets will come primarily from the
advance warning that politicians and managers
will be given of weak performance in particular
areas. This has the potential to improve
resource allocation to make it more likely that
key targets are met.

43
PMs for Public Policy Decisions
• Example Should policy A or policy B be
undertaken to reduce waiting lists?
• Current waiting list for an appointment at the
eye clinic 30 days.
• Contract pays 1 for the length of the waiting
list in days. And currently trades at 30 pounds.
• Participants in the market can BUY the contract
at 30 if they think the waiting list will
increase and SELL if they think it will decrease.
• E.g. If they SELL at 30 and the waiting list
decreases to 25 days, they will 5 (30-25). But
if the waiting list increases to 35 days, they
lose 5.
• By comparing the Waiting list with policy A
contract with the Waiting list with policy B
contract, the policy maker has gained information
on what the market thinks about the relative
impacts of introducing policy A and policy B on
the length of the waiting list.
• If a policy is not implemented, the contract is
declared void.

44
Using the power of prediction markets for disease
surveillance
• http//iehm.uiowa.edu/iehm/index.html
• Reporting speed is one of the most import
aspects of any surveillance program for seasonal
influenza even two weeks advance notice can have
dramatic results on the effectiveness of
vaccinations.
• Although there are many existing strategies for
gathering opinions about the future trends of
infectious diseases, the resulting data are often
difficult to interpret using standard
epidemiological methods. Prediction markets, on
the other hand, are well known for their ability
to quickly collect and summarize information.
• The Iowa Electronic Health Markets is a research
project at the University of Iowa exploring the
use of prediction markets as a tool for disease
surveillance. By combining the strengths of
prediction markets with the knowledge of our
trading community from around the world, our hope
is that these markets will report future
infectious disease activity quickly enough to be
clinically useful.

45
Limitations of crowd wisdom
• Can the crowd predict the lottery numbers?
• If not, why not?
• Because lottery numbers are drawn randomly, no
model or individual or crowd or other means of
aggregating information can predict them because
random numbers are by definition unpredictable.
• If the lottery numbers were, for whatever reason,
not drawn randomly, however, we have a different
issue.

46
Is Manipulation Bad for Prediction Markets?
• Robin Hanson and Ryan Oprea, of George Mason
University and the University of California,
Santa Cruz respectively, co-authored a paper
title, 'A Manipulator Can Aid Prediction Market
Accuracy. A perspective on its basic message is
offered by Alex Tabarrok at Marginal Revolution.
Tabarrok was considering the impact of the clear
attempt by at least one determined trader to
manipulate one of the US election betting markets
in favour of Senator John McCain. In particular,
the John McCain contract was bought in the
markets systematically every morning by one
US-based trader for sizeable sums. In
consequence, it was possible to arbitrage between
McCain (on Intrade) and Obama (on Betfair) for a
few weeks in the run-up to Election 2008.
• How much of a danger, Tabarrock asks, does this
sort of activity pose for the whole concept of
prediction markets? Not much, he argues, instead
offering support for Hanson and Oprea's finding
that manipulation can actually improve prediction
markets, for the simple reason that manipulation
offers informed investors a free lunch.

47
Manipulation (cont.)
• "In a stock market", Tabarrok writes, "... when
you buy (thinking the price will rise) someone
else is selling (presumably thinking the price
will fall) so if you do not have inside
information you should not expect an above normal
profit from your trade. But a manipulator sells
and buys based on reasons other than expectations
and so offers other investors a greater than
normal return. The more manipulation, therefore,
the greater the expected profit from betting
according to rational expectations.
• For this reason, investors should soon move to
take advantage of any price discrepancies thus
created within and between markets, as well as to
take advantage of any perceived mispricing
relative to fundamentals. Thus the expected value
of the trading is a loss for the manipulator and
a profit for the investors who exploit the
mispricing. Moreover, the incentive the activity
of the manipulator gives for others to become
informed, and to trade on the basis of this
information, is valuable in itself in improving
the efficiency of the market.

48
Worth manipulating?
• Tabarrok offers the additional observation that,
considerations of predictive accuracy aside,
there is one even more important lesson to be
learned from the activities of the manipulators
"...that prediction markets have truly arrived
when people think they are worth manipulating".
• But have they? What does the corporate sector
think?

49
HOW CAN COMPANIES USE PREDICTION MARKETS?
• To take an example, a manufacturer of aero
engines will seek good forecasts of future orders
from plane manufacturers, which in turn will be
contingent upon orders from airlines. Forecasts
of future airline orders will be greatly assisted
by the collation of a range of information from
those involved in each of these sectors.
• It is important that the questions posed are in a
form which is unambiguous and which can
ultimately be quantified. This requires an
assessment of who should be involved in
responding, and ensuring that each of these
contributors has an equivalent understanding of
the meaning of what is being asked, and that
these answers can usefully be pooled. The set-up
will vary depending on the diversity of
contributors, both geographically and
functionally. There is also the issue of
incentives and the number of markets to run, as
well as the length of these markets and how often
new markets should be introduced.
• But in principle markets should be able to help
aggregate information.

50
But whats the evidence? Can prediction markets
actually help internal company forecasting?
• There is in fact plenty of published research
showing how internal prediction markets have
helped improve the ability of commercial
organisations to structure and implement internal
prediction markets to assist in forecasting.
• to predict key business variables
• e.g. when will a product launch, what will be the
unit sales?
• broader-based prediction markets are a useful
mechanism for predicting market-wide outcomes,
e.g. box office receipts for a new film, success
of a new video game, property prices.

51
Commercial examples
• Eli Lilly ran an experiment in which managers
traded through an internal market the future
monthly sales figures for three drugs. The market
brought together all the information, from
toxicology reports to clinical results, and
produced more accurate forecasts than the
official forecasts.
• Google have set up a market, in which any Google
staff member could bet on the chances of an event
coming true. The markets were used to forecast
such things as product launch dates and new
office openings. The results - based on the
aggregated bets of thousands of Google staff
members - were strong predictors of the actual
outcomes.

52
How do these markets operate?
• Essentially, participants in the market exchange
offers and counter-offers until they agree on a
contract price.
• Trades are executed when two prices match.
• In describing how experimental internal-prediction
market run by Eli Lilly (the pharmaceutical
company) work, VP for Lilly Research Laboratories
Alpheus Beingham noted
• When we start trading in the drug and I try
buying your stock cheaper and cheaper, it forces
us to a way of agreeing that never really occurs
in any other kind of conversation.

53
Corporate Applications of Prediction Markets
Special Issue of the JPM
• The Journal of Prediction Markets (2009)
• www.thejpm.com
• Guest editor Prof. Koleman Strumpf, member of
the Editorial Board of the Journal of Prediction
Markets, and Koch Professor of Economics at the
University of Kansas School of Business.
• Based on presentations at the Conference on
Corporate Applications of Prediction/Information
Markets, held at Kansas Citys Kauffman
Foundation on 1 November, 2007.

54
What are Corporate Prediction Markets? Editorial
Introduction
• Prediction Markets use the knowledge of a pool
of individuals to help forecast questions of
importance to companies, such as whether a sales
target will be reached or whether a project will
be completed in a timely manner. A more recent
development is the use of such markets to
generate and evaluate new ideas, such as new
products or cost saving procedures.
• Since first being applied within the corporate
sector over a decade ago, over a hundred
companies have run internal markets. These range
in size from some of the largest in the world to
those with only a handful of employees, and cover
a broad range of sectors, from those whose
products are abstract ideas to others which
manufacture very low-tech products.

55
Why Have Such a Broad Range of Firms Become
Interested in Prediction Markets?
• The answer lies in a common problem facing
firms, namely the isolation of executives from
the views and insights of the companys
workforce.
• Such seclusion is no accident but instead
reflects one of the reasons companies are
structured as they are in the first place, i.e.
to avoid information overload for busy
executives.
• To reach this goal firms developed a hierarchy
structure, and assigned to middle management the
task of deciding how much and what information
was transmitted from employees to higher-level
decision-makers. But the system has its costs,
as potentially useful information may be filtered
out if it reflects poorly on those who control
the information flow. At the same time,
lower-level employees have little incentive to
make reports which conflict with their managers.
The net result is that executives may only
receive one-sided information, and flawed
decisions may result.

56
This is where Prediction Markets Come In!
• Suppose the CEO must decide whether to continue
funding a project, but is concerned that he has
been receiving overly optimistic reports on its
prospects from managers who will benefit from the
project continuing.
• A market on the projects prospects would allow
front-line employees to convey more realistic
information, and they could do so without fear of
reprisal if the trading is anonymous.
• Prediction markets may also function better than
other approaches currently in use. For example,
group meetings are less likely to have frank
discussions while suggestions boxes do not scale
well - prediction markets tend to perform better
when there are more participants.
• And while most workers may dread the thought of
meetings, markets are often considered a fun
activity and often do not require much in the way
of incentives to generate active employee
involvement.

57
A Case Study of GE (JPM, 2009), by Brian Spears
(Risk Manager, GE-Hitachi Nuclear Energy) et al.
• 1. Internal markets used to aggregate opinions
are consistent with opinions collected via web
surveys (Chan et al, 2002). Markets may in fact
improve upon traditional survey methods by
encouraging greater honesty from the
participants, providing participants with
valuable feedback from other participants, and
offering participants the joy of competitive
play.
• 2. GEs markets are designed to help answer
business questions such as What new technology
ideas should we be investing in? What new
products should we be developing?
• 3. Market participants can submit their own ideas
for entry into the market, and they can buy and
sell shares of any idea in the market.

58
GE Case Study (cont.)
• GEs interest in idea markets stem from our
belief that innovative new product and service
ideas can come from anywhere within an
organization.
• Similar to most companies, GE uses a variety of
methods to generate and down-select new ideas.
Traditional means include suggestion boxes and
brainstorming sessions.
• However, suggestion boxes often go unused because
contributors receive little or no feedback about
their idea or visibility into others ideas.
• Brainstorming sessions are often infeasible for
soliciting ideas from large, globally distributed
teams with potentially thousands of contributors.
• Hence the idea of a prediction market, which they
call an Imagination Market was born.

59
Conclusions of Case Study
• Overall, the GE Energy business was extremely
pleased with the results of the Imagination
Market.
• Funding was immediately provided to kick-start
the two ideas tied for the top, and the business
has decided to file patents for several others.
• GE Energy plans to continue use of markets in the
future.
• The volume and quality of ideas compared
favourably to brainstorming sessions, on-line
suggestion boxes, and on-line discussion forums.
• NB The Spears paper provides a wealth of
information and detail about the GE markets,
including a very detailed description of the
mechanics of the markets, such as the incentives
given to traders and to creators of new ideas.

60
Improving Forecasting Accuracy in Corporate
Prediction Markets A Case Study in the Austrian
Mobile Communication Industry JPM, 3, 2009, by
Martin Waitz and Andreas Mild
• ABSTRACT
• Corporate prediction markets forecast business
issues like market shares, sales volumes or the
success rates of new product developments.
• The improvement of its accuracy is a major topic
in prediction market research ...
• We propose a method that aggregates the data
provided by such a prediction market in a
different way by only accounting for the most
knowledgeable market participants.
• We demonstrate its predictive ability with a real
world experiment.

61
Why companies are well positioned to utilize the
information generated from PMs
• 1. Company divisions often serve as standalone
silos, and markets can be a means of integrating
the pockets of information contained in each.
• 2. Executives may be interested not just in
market aggregates, such as prices, but also the
trades of particular groups of employees. For
example, one could examine whether members of
certain divisions are less prone to making biased
forecasts.
• 3. Companies need real-time information about the
many uncertain events surrounding their
decision-makers.
• 4. Firms can internalize the informational
benefits of the market. A company can profit from
the information generated from prices, since the
market can be kept private and outside of the
purview of competitors.
• The last point is particularly important. Since
the benefits of the markets largely accrue to the
company, we might expect many prediction market
innovations to first arise in a corporate
setting.

62
Key challenges facing PMs
• 1. Operators must overcome investor reluctance to
a project with upfront costs and possibly delayed
benefits.
• 2. There are impacts on employees, both
detrimental (markets may distract staff away from
their main responsibilities) and beneficial
(there is often a gain in morale, as workers feel
empowered because their market-mediated
suggestions are impacting corporate decisions.
• 3. The markets may overwhelm executives with too
much information.
• 4. Market organizers must allay concerns of
middle management and those whose current role in
the company is threatened by the market.
• 5. There may be systematic biases in some markets.

63
Optimism bias
• Optimism bias is the systematic tendency for
people to be over-optimistic about the outcome of
planned actions. This includes over-estimating
the likelihood of positive events and
under-estimating the likelihood of negative
events".
• David Armor and Shelley Taylor highlight a number
of examples of what they consider to be optimism
bias in an interesting paper called 'When
Predictions Fail The Dilemma of Unrealistic
Optimism', published in 2002.
• Examples include students' estimates of the
likely starting salary of their first job in the
graduate market and newlyweds' thoughts on how
long their marriage will last. It is interesting,
therefore, that evidence of the existence of this
very same bias has been identified in 'internal'
company prediction markets, notably in a 2008
paper co-authored by Bo Cowgill, of Google,
Justin Wolfers of the Wharton School and Eric
Zitzewitz, based at Dartmouth College.

64
Optimism bias (cont.)
• Cowgill, Wolfers and Zitzewitz examine the
results generated by what they call the Google
corporate prediction market experiment. The
primary goal of these markets is, as they put it,
to generate predictions that efficiently
aggregate many employees' information and augment
existing forecasting methods.
• In support of previous investigations into the
value of internal prediction markets, they were
able to confirm that prices in the Google markets
closely approximated event probabilities, i.e.
that the markets were reasonably efficient. Even
so, they were not perfect, and one notable reason
was an apparent 'optimism bias' which, according
to their findings, "was more pronounced for
subjects under the control of Google employees,
such as whether a project would be completed on
time or whether a particular office would be
opened."

65
Optimism bias (cont.)
• Optimism bias was also found to be more evident
in new employees and in the immediate few days
following a good news day for the Google stock
price. Still, what is a cost in terms of
unadjusted predictive efficiency may be a benefit
in terms of motivation and entrepreneurial zeal,
a feedback mechanism the value of which it is
perhaps easy to under-estimate.
• In any case, if we are able to identify and
measure the source and extent of the bias, it
should be possible to adjust and compensate for
this particular inefficiency in generating the
objective forecasts.

66
The Favourite-Longshot Bias
• Let the probability of an event occurring be 20.
• Standard approach Probability 0.2
• Bookmakers approach Odds 4 to 1. This means
you win 4 (net) from the bookmaker if your bet
wins for every 1 staked (risked) with the
bookmaker.
• Which yields the better expected return, a stake
of 10 on a horse with odds of 2 to 1 or a stake
of 10 on a horse with odds of 20 to 1?
• i.e. if Mr. A and Mr. B both start with 1,000.
Now Mr. A places a level 10 stake on 100 horses
quoted at 2 to 1 and Mr. B places a level 10
stake on 100 horses quoted at 5 to 1. Who is
likely to end up richer?

67
Odds Backing Ladbrokes Pocket Companion, Flat
Edition, 1990, pp. 242-243
• Not one out of 35 favourites sent off at 1/8 or
shorter (as short as 1/25) lost between 1985 and
1989. This means a return of between 4 and 12.5
in a couple of minutes, which is an astronomical
rate of interest... The point being made is that
broadly speaking the shorter the odds, the better
the return. More broadly, the group of white
hot favourites (odds between 1 to 5 and 1 to 25)
won 88 out of 96 races for a 6.5 profit. The
following table looks at other odds groupings.
• Odds Wins Runs Profit
• 1/5-1/2 249 344 1.80 0.52
• 4/7-5/4 881 1780 -82.60 -4.64
• 6/4 -3/1 2187 7774 -629 -8.09
• 7/2-6/1 3464 21681 -2237 -10.32
• 8/1-20/1 2566 53741 -19823 -36.89
• 25/1 -100/1 441 43426 -29424 -67.76

68
An Election Super-Bias?
• The 2004 US Presidential state-by-state markets
gave the equivalent of 50 successive winning
favourites at the racetrack.
• In 2008, the Betfair favourites won 49/50 states.
• The Intrade favourites won 49/50.

69
Biases
• Do biases differ between different prediction
market formats?
• Can we compensate for the biases to yield more
accurate forecasts?
• Do some formats yield more volatile forecasts
than others?

70
Some further examples of PMs
• Best Buy, the electronics retailer. Experimented
with prediction markets on everything from demand
for digital set-top boxes to store opening-dates.
• E.g. in Autumn 2006, the price in one of their
PMs on whether a new store in Shanghai would open
on time several weeks ahead dropped sharply from
80 a share to about 45. Players made yes-no
bets, and the virtual dollar drop reflected
increasing doubt that the store would open on
time. The store opened a month late.
• Jeffrey Severts, a VP who oversees PMs at Best
Buy, quoted in NY Times (April 9, 2008 Betting
to Improve the Odds)
• The potential is that prediction markets may be
the thing that enables a big company to act more
like a small, nimble company again.
• It helps on two fronts, the speed and accuracy
of information, so that management can move
faster to deal with problems or exploit
opportunities.

71
• Severts invited several hundred employees to
submit an estimate for sales for a one-month
period. To help them calibrate their estimate, he
provided monthly data from the past twelve
months. 192 employees responded, including those
on the store floor. The estimates were given
equal weight and averaged. He found that the
employees collective wisdom had an error of only
0.5 compared to an error of 5 by the
• Severts went on to experiment with total sales
over the 14 week holiday period. He provided last
years sales figure from the holiday period and
revenue growth for the first three months of the
current fiscal year compared to previous years.
The original 350 respondents predicted sales
during the fourteen-week holiday period that was
99.9 accurate. The merchants themselves who were
traditionally responsible for forecasting were
93 accurate.
• (Hamel, 2007, The Future of Management, Boston
Harvard Business School Press).

72
PREDICTION MARKETS AS A MEDICALFORECASTING TOOL
DEMAND FOR HOSPITALSERVICES David Rajakovich
• This paper presents the outcome of a study
conducted at the Royal Devon and Exeter Hospital
in which a prediction market was established in
order to forecast demand for services.
• The study was conducted over a period of one
week, and involved 65 participants. Each was
asked to provide an estimate for demand for
services at the Royal Devon and Exeter Hospital.
In each survey, each employee was asked to
estimate the number of patients that would be
admitted to each directorate, which meant that
employees within each directorate were estimating
the number of patients admitted to directorates
other than their own.

73
Findings
• The overall results confirmed the effectiveness
of prediction markets.
• The prediction for admittances was 1157.51 while
the actual number of admittances was 1154, which
is an error of only 0.3.
• Market participants were almost exactly right in
the Medicine directorate, predicting 353.38,
while the actual was 353.
• Specialist Surgerys prediction was almost as
accurate with an actual number of admittances of
106 and an estimate of 107.75.
• However, the prediction market was less
successful in predicting demand for services for
each department, which the paper attributes to
the small sample size and lack of diversity of
participants.

74
Further examples
• In 2007, a group in the purchasing unit at
Hewlett Packard began prediction markets on the
price of computer memory chips three and six
• Bernardo A. Huberman, director of the social
computing lab at Hewlett-Packard The prediction
markets were up to 70 more accurate than the
companys traditional forecasting model ... The
more accurate predictions can be used to finesse
purchasing, marketing and product pricing
decisions.

75
The Classic Study
• Information Aggregation Mechanisms Concept,
Design and Implementation for a Sales Forecasting
Problem, by Charles Plott (CalTech) and Kay-Yut
Chen (Hewlett-Packard Laboratories), 2002.
• Many business examples share the following
characteristic small bits and pieces of relevant
information exists in the opinions and intuition
of individuals who are close to an activity. Some
examples are supply chain management issues,
demand forecasting, new product introduction, and
supply uncertainties. In many instances, no
systematic methods of collecting such information
exist. In these cases very little is known by any
single individual but the aggregation of the bits
and pieces of information might be considerable.
For instance, it is extremely difficult to
combine subjective information such as the
knowledge of a competitors move with objective
information such as historical data. In a perfect
world, with unlimited time and resources, a user
of such information could personally interview
everyone that might have a
• relevant insight but such luxury does not exist.

76
Chen and Plott (cont.)
• Gathering the bits and pieces by traditional
means, such as business meetings, is highly
inefficient because of a host of practical
problems related to location, incentives, the
insignificant amounts of information in any one
place, and even the absence of a methodology for
gathering it. Furthermore, business practices
such a quotas and budget settings create
incentives for individuals not to reveal their
information. The principles of economics together
with new technologies that exist for creating
markets and related mechanisms suggest that in
might be possible to develop a new approach that
avoids many of the practical problems.

77
Some details of the experiment
• The experiments were conducted with three
different HP divisions. Typically, around 20-30
people signed up for the experiments. Trading was
done through at a web server located at Caltech.
The subjects were geographically dispersed in
California.
• Typically, the prediction was for monthly sales
for a month three months in the future. The
market mechanism employed to support the markets
was the web based markets of the Marketscape
software, which was
• developed at the Laboratory of Economics and
Political Science at Caltech. All the markets for
an event were organized on a single web page for
easy access.
• A participant could enter a buy offer, a sell
offer or acceptance of an offer through the web
form on the page. Orders were compared to the
other side immediately. The best offers were
listed on the main market web page. The whole
book of offers was available for each market at
the click of a button.

78
(No Transcript)
79
Results
• Market predictions based on IAM (Information
Aggregation Mechanism) prices outperformed
official HP forecasts.
• In events for which official forecasts were
available the IAM predictions were closer to the
actual outcome than the official forecast 75 of
the time. The absolute errors of the official
forecasts were also significantly higher than
that of the IAM predictions.

80
Other general applications of PMs
• Examples of the applications of prediction
markets range from traditional finance
forecasting such as sales and costs, to product
development support such as forecasting on-time
project delivery or the likelihood of regulatory
approval for new drugs, to innovative decision
support such as evaluating the impact of
switching advertising agencies or forecasting the
market receptivity of new software releases
• Etc
• Etc
• Etc.

81
Its not just about forecasting!
• A prediction-market pilot at Microsoft in 2005
was designed to forecast the probability of
on-time release for several products.
• To managements surprise, the stock price
representing on-time release dropped to zero,
despite the staffs prior assurance that on-time
release was likely.
• The ensuing conversation uncovered the true
beliefs of the programmers, a result perhaps even
more valuable than knowing whether the release
would be missed.
• Source Todd Proebsting, of Microsoft Research,
in his presentation Tee Time with Admiral
Poindexter, delivered at the DIMACS Workshop on
Market as Predictive Devices (Information
Markets), February, 2005, Rutgers University.

82
Conclusion
• Prediction markets offer major unexploited
opportunities to aggregate information in a rapid
and efficient manner. There are significant
examples of success where they have been tried,
although in some cases they have been tried and
• In an interview with DIRECTOR magazine, I said
this
• The real problem where these markets have not
worked as well as management expected is that the
companies have simply bought the technology and
more or less expected it to take care of itself.
I dont want to stretch the point but this can be
compared to buying a car and expecting it to
drive itself. Of course youd be disappointed
with performance.
• And yes cars have design and performance
glitches, as do PMs.
• But cars can be useful and wonderful things.
• The same, may I say it, goes for Prediction
Markets.

83
Leighton.Vaughan-Williams_at_ntu.ac.uk