According to Brandon Nyhan at Dartmouth, well yes indeed they should.
Specifically:
“Under this approach, researchers would formally specify theoretical expectations, research design, and analysis plan before conducting a study. The act of committing to a set of predictions and hypothesis tests allows readers to better separate confirmatory from exploratory analyses and prevents scholars from Hypothesizing After Results are Known (HARKing). Researchers can of course still present analyses that were selected after examining the data in this framework, but they must disclose which results were not predicted in advance.”
The interviewer (associate professor Dan Hopkins from Georgetown) then replies something along the lines of “basically, you want researchers to do..what they do.”
But Nyhan says there is an even better way to approach the problem:
“I think a better approach is to offer a publishing option in which journals would consider accepting some articles in principle before the results were known based on peer review of the design and analysis plan. Such an approach, which has been formalized by the Registered Reports movement (of which I am a part), would better align author and journal incentives with our goals as scientists.”
Prof Hopkins then suggests that research costs are getting very expensive and wonders how pre-registration would affect that, and Nyhan replies:
“;;it’s easier than ever to mine experimental data from low-cost sources like Mechanical Turk where these constraints are much less likely to be binding.”
It is an interesting article and an idea worth considering, although somewhat problematic to request already-overworked good Peer Reviewers to review the methods (although this is basically my favorite part of a study). It also makes me wonder what is to stop someone from doing a study and THEN pre-registering the most interesting hypotheses that they get from playing with the data.
MTurk for Academics 