Crowds perform better than individuals

This new study is really interesting and multifaceted. They used MTurk, and found ways to identify how some individuals perform better than experts as well. HEre is the abstract:

“We analyze how 208 experts forecast the results of 15 treatments involving monetary and non-monetary motivators in a real-effort task. Wecompare these forecasts to those made by PhD students and non-experts: undergraduates,MBAs, and an online sample. We document seven main results. First, the average forecastof experts predicts quite well the experimental results. Second, there is a strong wisdom-of-crowds effect: the average forecast outperforms 96 percent of individual forecasts. Third,correlates of expertise–citations, academic rank,field, and contextual experience—do not

improve forecasting accuracy. Fourth, experts as a group do better than non-experts, but
not if accuracy is defined as rank ordering treatments. Fifth, measures of effort, confidence, and revealed ability are predictive of forecast accuracy to some extent, especially for non-
experts. Sixth, using these measures we identify ‘superforecasters’ among the non-experts who outperform the experts out of sample.

Cite:

Predicting Experimental Results: Who Knows What?
Stefano DellaVigna
UC Berkeley and NBER
Devin Pope
U Chicago and NB

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s