For some reason this opinion piece popped up in my Google Alerts yesterday–it is a bit old but suggests that researchers are ruining academic research by using MTurk since workers are non naive and thus can’t be fooled and so stuff isn’t being replicated:
“In short, psychologists and other social scientists are now facing a tragedy of the commons: We have exploited the online subject pools for our immediate personal benefit and now the pool of naïve subjects is depleted. This compromises our ability to use online platforms like MTurk to study many psychological phenomena, and prevents them from offering a powerful solution to the need for replication.”
It ends with an interesting call for researchers to get new people to join Mturk. How?
“Post a slide about signing up for MTurk or these other platforms in your class lectures.”
Because yeah, we want MTurk to look just like classrooms.
Anyway, I’m feeling a bit snarky today but here’s a suggestion–recruit workers who have completed fewer than 100 studies. And then complain that they don’t read the directions and miss the attention checks. But hey, you’ll get a naive worker!