Does the time a HIT is posted affect the subject pool?

A new study by Kiran Lakkaraju asks that interesting question . And the answer: well, not really!

Amazon Mechanical Turk (AMT) has become a powerful tool for social scientists due to its inexpensiveness, ease of use, and ability to attract large numbers of workers. While the subject pool is diverse,there are numerous questions regarding the composition of the workers as a function of when the “Human Intelligence Task” (HIT) is posted. Given the “queue” nature of HITs and the disparity in geography of participants, it is natural to wonder whether HIT posting time/day can have an impact on the population that is sampled. We address this question using a panel survey on AMT and show (surprisingly) that except for gender, there is no statistically significant difference in terms of demographics characteristics as a function of HIT posting time.

The study of more than 1000 Turkers showed that regardless of when a HIT is posted, the demographic profile of respondents will be similar with few exceptions: one exception is that you’ll get a lot more women on Friday and Saturday and lots more men on Thursdays and Sundays.

An interesting study, and it paid $1.00 for answering 120 questions.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s