Challenges to MTurk research

This blog post questioned recent research about American’s opinions about drones. TL:DR: research is flawed.

The blog author dumps on research done by Sarah Kreps in particular. His blasting of MTurk is below:

Though I like and respect Prof. Kreps, I find I must disagree with her research in this instance for a number of reasons. For example, in conducting her survey she employed an Amazon Mechanical Turk, which is an online service where people who self-select themselves to be employed as “workers” have the option to further self-select “Human Intelligence Tasks” in which to participate (in this case a survey) and to be paid.

Frankly, whatever value the “Amazon Mechanical Turk” may have for other “Human Intelligence Tasks,” I personally am very skeptical of its value in this particular context. Among other things, I think that the mindset of someone who decides to become a paid Internet “worker” is not necessarily coterminous with that of the average American, especially when the survey depends upon such “workers” choosing to be employed as a respondent on this specific survey.

I had to look up coterminous, and it means having the same boundaries or extent in space, time, or meaning. I’m not sure exactly what the author’s point is, as apparently he has a problem with a. people being paid to b. complete surveys, but again, I’m not sure what the particular context is that he has a problem with.

An abstract of Krep’s paper is here.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s