There are inevitably times in the career of any academic when an original hypothesis is not supported by subsequent research findings. In the past this has often meant that such findings went unpublished and did not therefore contribute either to personal advancement or departmental research rankings. All that has now changed. The European Journal of Negative Research Findings positively welcomes research papers which are unable to reach any conclusion whatsoever… Laurie Taylor Guide to Higher Education
There is no European Journal of Negative Research Findings. But, as it turns out, there is a Journal of Negative Results in Biomedicine, a psychology journal called Journal of Articles in Support of the Null Hypothesis as well as Journal of Negative Results: Ecology and and Evolutionary Biology.
Should there be an Economics Journal of Negative Research Findings? The publishers of the Journal of Articles in Support of the Null Hypothesis argue that there is a publication bias against negative research findings:
journals and reviewers have exhibited a bias against articles that did not reject the null hypothesis. We seek to change that by offering an outlet for experiments that do not reach the traditional significance levels (p < .05). Thus, reducing the file drawer problem, and reducing the bias in psychological literature. Without such a resource researchers could be wasting their time examining empirical questions that have already been examined. We collect these articles and provide them to the scientific community free of cost.
A number of studies (for a recent survey, see here) find evidence of such publication bias economics. There are two sources of bias. First, when a study gets the "wrong" result, e.g.an increase in minimum wages increases the quantity of labour demanded, the temptation is to say "there must be something with the study" and reject it. Second, negative findings are boring – who wants to know that variable X has no significant impact on Y?
Would the existence of an Economics Journal of Negative Research Findings diminish the extent of publication bias? Interestingly, although the Journal of Negative Results in Biomedicine has been in existence since 2002, it only publishes about a dozen studies a year, and the Journal of Articles in Support of the Null Hypothesis generally has no more than two or three articles in its semi-annual issues.
The lack of articles in these "negative findings" journals suggests that publication bias is coming from the supply side, from authors not submitting negative findings, as well as, or perhaps instead of, the demand side.
Perhaps one of Laurie Taylor's other inventions, The Comparative Journal of Overnight Articles ("two editions a day and a team of independent referees from all the major disciplines on constant standby") might have greater chance of success?
(HT to Linda, who told me about JASNH)
Worthwhile Canadian Initiative 
I don’t know about econ, but in the sciences, the negative results bias seems to come from the supply side of research funding. At the end of the day, you have to work that grant-funded research into a peer-reviewed paper, and journals don’t like to publish negative results. Grant committees aren’t likely to be impressed by citations in the journal of negative results.
I suppose the other source of “supply side” bias is that, if as you suggest, the temptation on getting a “wrong” result is to say that “there must be something wrong with the study”, authors may tweak their methodology to “fix” the study to get something other than a null result. To be clear, I’m not suggesting that this is a conscious bias, or that the “fix” isn’t done in the good faith belief that it will help arrive at a more accurate result (rather than just a “correct” result), rather I think this a probably a case of Coase’ adage that “if you torture the data long enough, it will confess”.
I lean towards the first commenter’s explanation. Journals are a two-sided market. If academics don’t want to read negative results, and by extension an academic’s reputation would suffer from publishing negative results, then it’s demand on the other side of the journal that’s driving the apparent supply side bias.
If we’re literally talking about failure to reject the null hypothesis, there’s a good reason not to publish the results; they are not informative. If we reject the null hypothesis, we can be pretty confident that it is false (e.g. 95% or 99% certain.) If we don’t reject the null, we really don’t know whether the null or the alternative is true. Furthermore, it is typically not hard to find some test with low enough power that we can accept any null.
“Laurie Taylor’s other inventions, The Comparative Journal of Overnight Articles”
In layman’s terms we call that a newspaper.
Simon: “If we’re literally talking about failure to reject the null hypothesis, there’s a good reason not to publish the results”
It’s interesting to contrast that journal with the Journal of Negative Results in Biomedicine, which has the following submission guidelines:
Articles published in traditional journals frequently provide insufficient evidence regarding negative data. They hardly allow a rigorous evaluation of the quality of these results. In addition, controversial results that refute a current model or simply negative results within a current dogma, frequently meet considerable resistance before they are acknowledged. This is particularly the case if current techniques and technologies are too crude to shed further light on the findings.
Then again the seriousness of a JNEB publication is also guaranteed by the $1685 submission/publication fee!
One should note that the FDA now insists on a list of studies done before drug trials are conducted and all data must be reported. The simple fact of the matter is I can randomly distribute 101 variables and 5 of them will correlate with the first at 95% confidence. These “N trial” effects must be taken into account. If only positive results are published, then accidental correlations will show up in the literature. Without the negative results you get all sorts of madness.
In math, the Hackmem was a document that contained some negative results.
So JNEB is a vanity press. 1) author pays a lot; 2) accepts papers that would be rejected by actual reviewers.
Jon “So JNEB is a vanity press.” – actually, this level of submission fee is not unusual for an open-access journal in the sciences – the idea is that the journal is funded out of researchers’ grant money, rather than through library subscriptions.
Great post!
I think biais is going down though. I think writting paper is an art. I know one AP at a top 10 who has 3 negative papers published: 2 in top5 (QJE, JPE) and one in top field.
He writes paper like Does X affect Y? and his questions are very good. He found no significant results where there should be and tried to explain why. He used a top notch econometric technique (RDD or DD…) and found a reasonable explanation of why no effect. When you read his paper, his contribution is very clear. I think he might get tenure with this…
I think maybe now journals are more open to this but researcher still think it is not possible… It is very risky to try to go ahead with such paper while chasing tenure… or a job for grad students…
On a related note, I think we would need a journal called: Journal of replication studies. I think replications are not valued enough in the profession. It is in my opinion very important…
Quoting: “There is no European Journal of Negative Research Findings” I think this is not true, I have found The All Results Journals (http://www.arjournals.com) that is being published by SACSIS a Spanish not-profit organization. They publish negative results in several areas (Chemistry, Biology, Physics and related) and do not charge authors to publish (they state in their website they are the first Total Open Access journals).
Have a look at their press release for more info:
http://cordis.europa.eu/wire/index.cfm?fuseaction=article.detail&rcn=25891&rev=0&PageMode=print