Turkers aren’t as sleepy as students

That’s one of the conclusions of a new study comparing MTurk workers to students and a community sample: “MTurk participants were less likely than campus and community participants to complete studies while sleepy “. The students are at the University of Chicago and the community is Chicago. Some interesting things from this study:

-Turkers were paid .75 for the study (it took about 7 and a half minutes), students and community members were paid $3 (or course credit for students). According to the authors: “although this is nearly twice the rate that MTurk participants were paid, this payment discrepancy reflects the typical market rate for participation compensation for each of the samples and is common in similar designs which compare MTurk to other samples.” They said that this rate was $12 per hour so I guess it took both students and community people twice as long to fill out the survey? Because it seems like four times to me.

-According to the study: “MTurk participants are somewhat more distracted than participants from non-crowdsourced samples—they are more likely to multitask during studies and to leave the page of a study while they are completing it.” Yes, but this is compared to what I think are people (students and community members) taking it in a lab.

-Also, “Somewhat troublingly, MTurk participants also report that they participate in studies by researchers that they already know more often than do participants from the campus and community.”  The researchers think this is troubling because Turkers are taking the same survey multiple times. However, I think this is due to Turkers checking out researchers on the Turkopticon before they decide whether to do a study.


-And confirming every study every done: “We also observed that participants who completed more studies generally reported less frequent engagement in potentially problematic respondent behaviors, consistent with what would be predicted by Chandler and colleagues’ (2014) [15] findings that more prolific participants are less distracted and more involved with research than less prolific participants.”

Cite: Necka, Elizabeth A., Stephanie Cacioppo, Greg J. Norman, and John T. Cacioppo. “Measuring the Prevalence of Problematic Respondent Behaviors among MTurk, Campus, and Community Participants.” PloS one 11, no. 6 (2016): e0157732.




Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s