This new article provides some historical context for analyzing crowdwork like that which occurs on MTurk. It looks at the use of piecework in manufacturing and draws some interesting implications. Some of these key implications include:
- Complexity: when something moves from a simple and familiar task (sewing) to one that is more constrained (sewing to the reqs of a manufacturer), masurement and verification will remain persistent challenges that will limit complexity unless solved.
- Decomposition: at some point, specialized training for work will become the norm.
- Workers: the decentralized nature of workers will continue to limit collective action.
This is a great foundational piece for anyone doing MTurk research.
Yep, it can be done.
“We investigated whether the online platform, Amazon Mechanical Turk (MTurk), could be used as a resource to more easily recruit and measure the behavior of infant populations. Using a looking time paradigm, with users’ webcams we recorded how long infants aged 5 to 8 months attended while viewing children’s television programs.”
Here’s the procedure:
” After accepting the HIT, participants were directed to a webpage that provided an information sheet and asked for their informed consent. Consent was obtained via online checkbox and button press. This was followed by an evaluation of the suitability of participants’ computers, software, webcams, speakers, and internet connectivity. To do this, we recorded a brief 5-s video of caregivers and their infants and asked them to move and make sounds. This video was then played back to caregivers, and they were asked to check a box to indicate whether or not they were able to see and hear themselves. If they indicated they could not, they were thanked and excluded from participation. Otherwise, they were directed to a new webpage instructing them to position their infants on their laps in the center of the screen and in a well-lit room. This was specified to ensure that infants’ eyes were visible while recording the webcam videos. Once they were in a comfortable position, caregivers were instructed to press a “start” button to commence the experiment. Then 1 of 10 pseudorandomly selected movies was presented. Afterward, participants completed a short demographic questionnaire from which the ages and language backgrounds of their infants were obtained.”
Read it all here: http://www.sciencedirect.com/science/article/pii/S0022096516302909
Panos Ipeirotis posted his thoughts as to why prospective workers are getting rejected, read the whole thing here.
Bottom line: if Amazon can’t verify who you are financially (you don’t have a credit record or bank account, or if you fail some type of CIP verification).
Keep in mind, though, Amazon probably won’t ever tell us.
A new study investigated whether people who are addicted to alcohol, marijuana and gambling can be effectively studied using MTurk. The answer?
“The results suggest that the self-report data obtained from alcohol and gambling populations are of high quality, however, caution is warranted with cannabis populations.”
Those last four words are Words to Live By.
Kim, H. S., & Hodgins, D. C. (2016). Reliability and Validity of Data Obtained From Alcohol, Cannabis, and Gambling Populations on Amazon’s Mechanical Turk. Psychology of addictive behaviors: journal of the Society of Psychologists in Addictive Behaviors.
This new study compares 2700 Turkers to the 1300 respondents to the CCES (the Cooperative Congressional Election Survey).
Some interesting findings:
*MTurk is relatively strong at attracting young Hispanic females and young Asian males and females.
*The number of respondents living in different geographic categories on the rural–urban continuum is almost identical in MTurk and CCES. Both MTurk and CCES draw approximately 90% of their respondents from urban areas.
The authors conclude that researchers can use these pools to over-sample and stratify to build samples that are balanced on theoretically motivated variables of interest.
And of course this doesn’t even get you up to minimum wage. But at least the majority support the highest amount.
So I guess this is a thing now?
Are YOUR aces just?
This story from CrowdCamp says that Turkers aren’t worried about AI taking over their Turk work–yet.
There’s a lot of other interesting info in this report as well.
This study uses a small sample and open-ended questions to examine MTurk workers, particularly motivations. The study shows three different motivations:
1.Financial needs: receiving a monetary reward for work. Financial needs were strongest needs for joining MTurk.
2. Personal growth needs: building and honing skills. Once people start working on MTurk, many find that the tasks help them grow skills.
3. Pro-social needs: contributing to society (such as through participating in research studies).
The second two motivations keep people coming back to work, according to the study.
Amazon really needs to step in and BAN THIS REQUESTER. This is nuts. Requesters have to pay upfront so they don’t have limited funds. This is just awful.