ACM paper provides Turk Tips for longer surveys

A paper to be presented at ACM in Atlanta and available for download here has some great tips for using MTurk and ensuring results are valid. The study itself is quite interesting, but the task was complex. Workers needed to learn how to use an interface to find out specific information, and once they learned it they were given 15 minutes to use the interface and find information. There was quite a bit of work that was part of this HIT.

Specific tips, particularly for a longer study:

-the researchers required participants had completed at least 50 hits
-participants were paid an average of $4 with a $1 bonus paid to people who completed the study
-the researchers used numerous satisficing checks (19!)
-the researchers providing a brief training phase–more than 500 people started the training, and just about 200 completed the actual HIT.

As we know from previous posts, $4 (or $5 with a bonus) for 15-16 minutes work is quite acceptable, so bravo to these Computer Science researchers from UCSB, U of Illinois and SA technolgoies.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s