Prediction Market Angst
For those of you who aren’t prediction market enthusiasts, there is a vibrant community that has developed - much of it manifested online - over the past few years. Like many such emergent communities, there is much passion amongst the members for there chosen area of focus. Generally this is a good thing but is not immune to excess from time to time as some of this passion occasionally splills over into what might fairly be described as zealotry. I guess I would consider myself to be a member of this community, probably somewhere towards the middle or the back of the room: trying to follow the main plots, absorbing quite a bit of information and occasionally asking a question or making an observation. So in this micro-context, on the spectrum from zealot to reactionary sceptic, I’d probably be halfway between the middle and sceptic; however if you choose the general population as your sample, I’d be close to the zealot camp. Everything is relative.
But - much as I entertain a similar passion for the potential power of markets in “predictions” as many of the leaders of this movement, I have to chuckle when I run across some of the more - shall we say “emotional” discussions within this community. Again while this is a common characteristic of most new ‘movements’, and in no way unique to passionate prediction market proponents, it is still funny to see this community’s People’s Judean Front and the Judean People’s Front equivalents bash it out on the airwaves so to speak.
And of course there is a small fringe that is keen to control the terms of engagement - the quintessential ‘rule-writers’, often obsessing about semantics and making sure that everybody innovates but only in the ‘right’ way. Another irony - from my point of view - is that many (perhaps most?) of the most active, eloquent and committed members are American (or more relevantly based in America), where of course the wise people in Washington, DC make it as hard as possible to run said prediction markets without risking 2-5 years… Actually, when you stop to think about it this is actually less surprising than at first glance: repression is a great catalyst to passion. (Thank goodness!)
Anyhow, last week’s Clinton victory in the New Hampshire primary has thrown this community into a tizzy…sort of like kicking an anthill…many many column inches have been written defending, deriding and analyzing the fact that the ‘(prediction) market(s)’ got it WRONG! ie Didn’t accurately predict the outcome - indeed they were no better than the POLLS! Oh the shame…
Well - and I suspect many others have already made the same observation so I make no claims to any original insight - this line of reasoning misses the point entirely in my opinion. The ‘failure’ of New Hampshire was the result of primarily two factors:
- It wasn’t a failure. No market is always right. More importantly markets reflect the information available to and the interests of their participants. Basically markets are very efficient mechanisms (I would claim the most efficient) for processing information. No more, no less.
- In this particular instance, the probability of the market producing an erroneous forecast was high due to the lack of liquidity. This is a problem of all political markets in the US. Show me a market on the New Hampshire primaries with tens of thousands of participants and millions of dollars traded and I will show you a market that creates more valuable information. BUT it would still on occasion be ’surprised.’
Basically I guess what I’m trying to say is the expectations seem to be set all wrong by many inside the community. I think “prediction markets” - creating markets in information and outcomes is a wonderfully important and valuable thing to do. Equally however I think that anyone that represents such markets as being able to predict the future is a charlatan. What they can do is collect and synthesize powerfully and efficiently all the dispersed available information - using money as the relevance filter. This is very valuable in its own right and is defensible. Promoting prediction markets to true sceptics (ie mainstream American politicians) on the basis that they are a Delphic Oracle is surely a path to certain tears and ultimately is almost guaranteed to fail.
Markets don’t compute unknown unknowns. That doesn’t mean they are useless, just that they have to be understood in context.
January 14th, 2008 at 3:00 pm
Incidentally Google just published a paper on their experiences of using prediction markets internally, for stuff like ‘estimate how many people will sign up to GMail by year end’.
http://googleblog.blogspot.com/2008/01/flow-of-information-at-googleplex.html
Their findings on correlation behaviours are probably highly relevant to anyone managing a large trading desk where traders’ proximity can lead to any number of (bad) cognitive biases and correlated (more risky) portfolios.
January 14th, 2008 at 4:18 pm
[…] Chris. F. Masse January 14th, 2008 Sean Park (after a long, boring introduction to the subject): […] The ‘failure’ of New Hampshire was the result of primarily two factors: […]
January 14th, 2008 at 5:33 pm
Thanks Alex, I had actually linked to the google story in one of my posts last week. I thought about expanding on the same idea you point out in a post “Why-trading-rooms-still-matter-in-a-digital-age…” but was too busy.
So yes agree.
January 16th, 2008 at 10:54 pm
The failure of the exit polls in NH is probably a lot simpler than you suggest (a. it wasn’t a failure or b. the statistical wasn’t high enough).
Exit polls are notoriously fickle - remember the UK Election of ‘92, where exit polls initially indicated a win for Labour, which ended up with the Major government winning?
Well it’s basically because people lie to exit pollsters.
There were two seperate phenomenom at work in NH - the first is based on race and is a well understood phenomena amongst political psephologists, called the ‘Bradley effect’. It basically states that prior to polling, voters mislead pollsters by declaring they will vote for the black candidate or are undecided, but when in the voting booth, veer towards the white candidate.
This probably explains a little of what happened in NH, but the real culprit is what is termed ‘the Spiral of Silence’ - people say they voted Obama when they actually voted Hillary. Exit polls bear this out, as women in particular voted for Hillary in record numbers over Obama, and are considered more likely to lie in front of their partners in front of an exit pollster. (no angle here! just a well-recognised trend in psephology!).
The as always excellent Danny Finkelstein covers it in his US Election blog here -
http://timesonline.typepad.com/comment/2008/01/more-on-why-hil.html
I was always taught in my US Politics class at University that America would never elect an african-american. Well, it seems in this election, socially at least, a black man is more acceptable to middle America than Hillary Clinton will ever be.
February 1st, 2008 at 12:23 pm
Sean,
A new market for “Search Engines” by Dr. David Pennock, Principal Research Scientist at Yahoo! and Intrade:
http://www.intrade.com/news/news_190.html
http://www.intrade.com/index.jsp?request_operation=trade&request_type=action&selConID=573275
Alexander