In which I compare MTurk to Qualtrics and Students and more

Well, it isn’t just me, and there isn’t much ‘more’, but a recent article I co-authored is available for free download through May. Just go here.


Data collection using Internet-based samples has become increasingly popular in many social science disciplines, including advertising. This research examines whether one popular Internet data source, Amazon’s Mechanical Turk (MTurk), is an appropriate substitute for other popular samples utilized in advertising research. Specifically, a five-sample between-subjects experiment was conducted to help researchers who utilize MTurk in advertising experiments understand the strengths and weaknesses of MTurk relative to student samples and professional panels. In comparisons across five samples, results show that the MTurk data outperformed panel data procured from two separate professional marketing research companies across various measures of data quality. The MTurk data were also compared to two different student samples, and results show the data were at least comparable in quality. While researchers may consider MTurk samples as a viable alternative to student samples when testing theory-driven outcomes, precautions should be taken to ensure the quality of data regardless of the source. Best practices for ensuring data quality are offered for advertising researchers who utilize MTurk for data collection.

And for your amusement, I have a colleague who hates MTurk. He wrote an anti-Turk article and it is available for free too, along with our response to him. You can find those pieces, along with several other pieces about different types of methodology, here:

I was just in Boston and I organized a panel about MTurk where it turns out the Colleague/Hater said that he ‘used to represent Qualtrics’ at academic conferences. I’m not sure exactly what he meant by that, but it sounds like there’s a bit of—oh I don’t know—biast? there.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s