This a finding from the same study I cited yesterday. It is the top five Requester actions that Turkers say contribute to low job satisfaction:
- Paying an unfair wage
- Including difficult attention check questions
- Using majority rules (inter-Worker agreement) for rejection decisons
- Advertising a HIT as taking less time than it actually does
- Blaming others (e.g. other members of one’s research team) for rejection and refusing to investigate issues further or consult the person who recommended rejection.
I certainly agree with #1 and #4. I haven’t had experience with #3. I definitely agree very very strongly with #5: OWN YOUR DECISIONS PEOPLE.
I have to admit I’m a bit stuck on #2. First, difficult is in the eyes of the beholder. Second, this is one of the key issues of validity that reviewers and journals are looking for. The article had an example ‘narrative’ to #2, and it read ““I didn’t summarize the experiment with the wording she wanted, and was rejected.” Hmm. That isn’t exactly an attention check question in my viewpoint.
Citation: Brawley, A. M., & Pury, C. L. (2016). Work experiences on MTurk: Job satisfaction, turnover, and information sharing. Computers in Human Behavior, 54, 531-546.