Citation bait? Study about

This has nothing to do with MTurk but everything to do with academic publishing: Scholarly Kitchen’s report on a new study about

I quote the report which quotes the study:

We find that a typical article posted to has 75% more citations than an article that is available elsewhere online through a venue: a personal homepage, departmental homepage, journal site, or any other online hosting venue.

And then Scholarly Kitchen EXPLODES OUR BRAINS (not really):

This paper suffers from a data problem. And that problem is their control group.

The researchers compared the performances of papers uploaded to to a control group–a random sample of papers selected from the same journals. If the randomization was successful, the control group should be similar in all respects to the group. In this way, differences observed in the data over time are likely attributable to and not some other cause.

If you look at their data (download the file: papers.csv.gz), you’ll notice something odd in the title of the first article: it is an errata and it belongs to the control group. Indeed, if you search the title list, you’ll find editorials, corrections, retraction notices, letters to the editor (and their responses), commentaries, book reviews, conference program abstracts, news, and even obituaries in the control group. As a general rule, these kinds of papers receive few (if any) citations. So, it’s no surprise that papers uploaded to outperformed a sample that included a good proportion of non-research material.

Data. So pesky.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s