Paid crowdworkers do best with simple tasks

That’s the conclusion of a new study:

“In this study, we conduct an initial investigation on the effect of crowd type and task complexity on work quality by crowdsourcing a simple and more complex version of a data extraction task to paid and unpaid crowds. We then measure the quality of the results in terms of its similarity to a gold standard data set. Our experiments show that the unpaid crowd produces results of high quality regardless of the type of task while the paid crowd yields better results in simple tasks. We intend to extend our work to integrate existing quality control mechanisms and perform more experiments with more varied crowd members.”


Borromeo, Ria Mae, Thomas Laurent, and Motomichi Toyama. “The Influence of Crowd Type and Task Complexity on Crowdsourced Work Quality.” In Proceedings of the 20th International Database Engineering & Applications Symposium, pp. 70-76. ACM, 2016.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s