Comparing MTurk to PA, CF and CBDR

How does MTurk compare to Prolific Academic, Crowdflower and the Carnegie Mellon poll (also known as CBDR)?

This new articles tells you.

CBDR was the ‘control’, so to speak, and comparing MTurk to PA and CF shows:

“In two studies, we found that participants on both platforms  (PA and CF) were more naïve and less dishonest compared to MTurk participants. Across the three platforms, CF provided the best response rate, but CF participants failed more attention-check questions and did not reproduce known effects replicated on ProA and MTurk. Moreover, ProA participants produced data quality that was higher than CF’s and comparable to MTurk’s. ProA and CF participants were also much more diverse than participants from MTurk.”

So it looks like a thumbs up to Prolific Academic.

Nice job from some very expert MTurk researchers!

Eyal Peer, Laura Brandimarte, Sonam Samat, Alessandro Acquisti, Beyond the Turk: Alternative platforms for crowdsourcing behavioral research, Journal of Experimental Social Psychology, Volume 70, May 2017, Pages 153-163, ISSN 0022-1031,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s