I asked two individuals very involved in the MTurk community, Kristy and Rochelle, about what they thought about the ‘academic work is different’ argument.
With their permission, here are their excellent thoughts:
- From Kristy: “Check out the Requester FAQ: https://requester.mturk.com/help/faq. We’re referred to as workers, not volunteers, not study participants. We are workers, and when we demand a fair wage for our labour, we are doing so because we’re here to make money. What if the person who oversees your research told you that you’d need to start doing it for less than minimum wage? You wouldn’t stand for it, and neither will we.”
- From Rochelle: ” see the (valid) point about how research studies should be unpaid/volunteer in order to get a true sample without the possibility of it being tainted by someone performing a specific way for money. I know for in-person lab studies, some are compensated and others aren’t. I personally was involved in one (not as a student – just as a public community member) and was only reimbursed for my parking fees. I see it the same way as how journalists don’t pay subjects they interview. It’s a conflict of interest and you’re not going to get a true, honest, unbiased response if the person is “hired” by the journalist to respond.However, the point of MTurk is to perform tasks for payment. When research studies started being posted on MTurk, they needed to have payments attached or else no one was going to do them. People are on MTurk to make money. Why would they choose an unpaid survey HIT over a paid non-survey HIT? Thomas Leeper brought this up on Twitter a few days ago: https://twitter.com/thosjleeper/status/560881540021104641″
- From Kristy: “If you want to compensate people ridiculously for their participation in research that will get you a degree or your paycheque, both of which are worth far more than HITs on mTurk, then go to another site. Be warned – even Prolific Academic (Kim’s note: this is a site being started in the UK for academic research), the best site for researchers to find participants online, insists you pay at least minimum wage for their time.”
- From Rochelle: I very strongly disagree with that 2011 study about MTurkers only participating for intrinsic motivation and not for actual money. That study is likely very outdated as conditions and situations change rapidly in this environment. The economy and world of crowdsourcing and digital labor was completely different in 2011.
- From Kristy: “Check out Litman, Robinson & Rosenzweig, 2014 and Shulze, Seedorf, Geiger, Kaufmann & Schader, 2011 which contradict Buhrmester et al.”
- From Rochelle: “Something else of note from that 2011 study: “We posted a task that paid workers 1 cent for answering two pieces of information: age and gender. In 33 hours, we collected 500 responses or about 15 participants per hour. These results demonstrate that workers are willing to complete simple tasks for virtually no compensation, again suggesting that workers are not driven primarily by financial incentives.” Something they didn’t consider was that workers, especially new ones, will often do low paying HITs in order to increase the number of overall submitted HITs they have. (Kim says: as a new worker, I can definitely confirm this is true). The more HITs you have completed, the more access you have to higher paying HITs. Plus, workers try to get a lot completed HITs done so that when they inevitably get a rejection, it won’t decrease their overall approval rate quite so much.(Kim says, again, as a new worker, there is nothing more disheartening than seeing your approval rate dip). If you have a lot of HITs completed, you’ll have more padding when a rejection occurs so your account won’t be quite so damaged. In this case, the worker may not be “driven primarily by financial incentives” but that doesn’t mean that they are doing the HITs for intrinsic motivation or for fun/pleasure only. They’re doing it as a necessary evil in order to get access to better HITs and as a method of account protection and insurance.”
Thanks to both Kristy and Rochelle for their detailed and insightful comments.