A new article looks at some of the key criticisms of MTurk for organizational and information systems research and does a great job of taking some of the critiques that are written about all. the. time and turning them into advantages.
Critique: “We don’t know who these people are and people aren’t who they say they are.”
Advantage: A researcher can use multiple screeners to identify people who aren’t who they say they are (if they even exist) quickly and inexpensively.
Critique: Super Turkers distort results.
Advantages: Responses will be higher quality because people won’t be confused by questions, are attentive, and don’t get tired of answering surveys.
Critique: People lie.
Advantages: People always lie (ok this was from Dr. House, not from this article). Use of attention checks with appropriate punishments for failing attention checks (ie not getting paid) helps to identify people who aren’t good participants.
Critique: People on MTurk may not be normal.
Advantage: And people who complain about MTurk are (ok that was from me, not from this article). The article says that paying at least minimum wage will help to attract more people to the site and to address any lack of normality.
It’s a good article and I encourage you to read it–I got the full piece from Researchgate.