Using MTurk for Content Analysis

A new study on Twitter engagement and diabetes > used Turkers to conduct a content analysis of tweets with the hashtag #diabetes.

To do the content analysis, they had each tweet coded for informational category (whether it was about people, an event, a success story, a failure story, etc) and sender description (person, organization, neither). They had each tweet coded by four different Turkers at a cost of 7 cents per tweet. It took on average about three and a half minutes to code a single tweet.

To show validity, they calculated the intraclass correlation coefficient for each topic and user types. Any topic or user type classification selected by two or more Turkers for a tweet was assigned to the Tweet. They found reliability was good or excellent with four coders.

I’m unfamiliar with the study by Hipp et al that they used as the framework (abstract here . It will be interesting to see if this method makes it way to the social sciences.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s