Skip to main content
About HEC About HEC Faculty & Research Faculty & Research Master’s programs Master’s programs MBA Programs MBA Programs PhD Program PhD Program Executive Education Executive Education Summer School Summer School HEC Online HEC Online About HEC Overview Overview Who
We Are Who
We Are
Egalité des chances Egalité des chances HEC Talents HEC Talents International International Campus
Life Campus
Sustainability Sustainability Diversity
& Inclusion Diversity
& Inclusion
Stories Stories The HEC
Foundation The HEC
Coronavirus Coronavirus
Faculty & Research Overview Overview Faculty Directory Faculty Directory Departments Departments Centers Centers Chairs Chairs Knowledge@HEC Knowledge@HEC Master’s programs Master in
Management Master in
Programs Master's
Double Degree
Programs Double Degree
Programs Summer
students Exchange
Life Student
MBA Programs MBA MBA Executive MBA Executive MBA TRIUM EMBA TRIUM EMBA PhD Program Overview Overview HEC Difference HEC Difference Program details Program details Research areas Research areas HEC Community HEC Community Placement Placement Job Market Job Market Admissions Admissions Financing Financing Executive Education Executive Masters Executive Masters Executive Certificates Executive Certificates Executive short programs Executive short programs Online Online Companies Companies Executive MBA Executive MBA Infinity Pass Infinity Pass Summer School Youth Programs Youth Programs Summer programs Summer programs HEC Online Overview Overview Degree Program Degree Program Executive certificates Executive certificates MOOCs MOOCs Summer Programs Summer Programs Youth programs Youth programs


Scientific Research: Should Negative Results Be Published?

Decision Sciences
Published on:

Many call for a systematic publication of scientific negative results in order to make the production of scientific knowledge more efficient. Raphaël Lévy, Assistant Professor in the Economics and Decision Sciences Department, explains why such dissemination of knowledge may actually be beneficial, but also points to potential undesired consequences.

old man writing stokkete-AdobeStock_cover

©stokkete on Adobe Stock

A major concern for inventors, scientists and researchers in general is to establish the primacy of their discoveries by being the first to disclose an advance in knowledge, via either a scientific publication or a patent. This race tends to accelerate innovation, at the cost of possible adverse effects on quality.

The dilemma: quality versus speed

During the development phase, researchers and inventors face a dilemma. On the one hand, a longer maturation increases the chance of detecting an anomaly which reveals that their project is a poor lead that should be dropped (1). On the other hand, the fear of being overtaken by a rival may hasten publication or marketing decisions. The tension between these two competing forces (more learning versus the risk of being preempted) is characteristic of any innovation process, and the timing strategy ultimately chosen by scientists determines the final quality of their innovation.


The tension between a longer maturation and the risk of being preempted is characteristic of any innovation process.


The issue of the publication of negative results in science

In a world where the quality of scientific publications is increasingly challenged, the public debate has focused on the mechanisms that foster high-quality research. The question of whether or not to publish negative results, in particular, has received much attention. Recent studies show that academic journals have a marked bias for positive results (e.g., results that are significant in a statistical sense) (2).

However, the fact that negative results (for instance, results which, conversely, have insufficient statistical significance) are not made public is potentially harmful in two respects. First, a negative result is a valuable source of information for other researchers. For example, if two researchers are trying to prove the same theory, and one of them finds a counter-example that proves the theory wrong, the other researcher could benefit from knowing this information, which could spare him the cost of pursuing the same direction in vain. Second, the non-publication of negative results creates a selection bias that produces a distorted overall picture of the state of the art (3).

©kasto on Adobe Stock

This is why many experts have recently spoken out in support of the systematic publication of negative results. For instance, the World Health Organization states that “negative and inconclusive as well as positive results must be published or otherwise made publicly available" (4). In line with this trend, several academic journals have decided to give greater publicity to negative results, either directly by promoting their publication or indirectly by requiring researchers to register their experimental projects in advance (5). 

The publication of negative results does not necessarily increase the quality of publications

What impact should we expect from such a systematic disclosure of negative results on the quality of publications, in particular in a highly competitive research environment? We address this question in an on-going project with Catherine Bobtcheff (Paris School of Economics) and Thomas Mariotti (Toulouse School of Economics), where we model the choice made by a researcher of the duration of the learning phase in two distinct scenarios. 

In the first scenario, negative results are systematically published and thus immediately made public: since the information is shared by the entire research community, each researcher learns faster than otherwise, hence an incentive to shorten the learning phase all the larger as competition is strong.

In the second scenario, negative results are never made public, and researchers respond to this by waiting longer to learn more about their projects. Indeed, they recognize that being the first to publish is now potentially bad news since a rival may have obtained a negative result and kept it private. To take up the earlier example, if a researcher who suspects that a competitor is trying to prove the same theory does not see the latter publish anything, he must take into account the possibility that the rival has found a counter-example that invalidates the theory, without this "negative result" being made public. This phenomenon well-known from the analysis of auctions is called the "winner's curse". It encourages researchers to prolong the learning period to enhance their confidence in their results.

Comparing the merits of these two scenarios highlights two opposite effects that the systematic publication of negative results has on quality. The learning process in the maturation phase is faster when the results are public (everyone benefits from more information), which is good for quality. At the same time, researchers adjust their timing strategies by shortening the learning period, which is detrimental to quality(6).


These preliminary findings already demonstrate that one should be careful before endorsing the systematic publication of negative results as a way to foster the quality of publications, especially in a highly competitive research environment.


By quantifying these two effects, our analysis shows that, when competition is strong, the second effect may be stronger, hence a lower average quality of publications when negative results are public! These preliminary findings already demonstrate that one should be careful before endorsing the systematic publication of negative results as a way to foster the quality of publications, especially in a highly competitive research environment.


(1) During this “learning phase”, a pharmaceutical firm typically performs additional tests on its new molecule to rule out the risks of side effects, or researchers carry out robustness checks on their results.
(2) See Fanelli, D. (2012), "Negative Results are Disappearing from Most Disciplines and Countries", Scientometrics, 90, 891-904.
(3) For example, a study by Turner et al. (2008), "Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy”, The New England Journal of Medicine, 358, 252-260, shows that 94% of the studies on anti-depressants considered by the Food and Drug Administration (FDA) to approve marketing decisions exhibit positive results. But the percentage of positive results falls to 51% when unpublished studies not taken into account by the FDA are included in the panel.
(4) See the results here.
(5) For example, the Journal of Development Economics gives to authors the opportunity to have their potential empirical projects reviewed and approved for publication before the results are known.
(6) For instance, if a drug actually has important side effects, the firm is less likely to find out because the testing phase is shorter, so the risk that the drug is inappropriately brought to the market increases.


Related content on Decision Sciences


How Much to Reveal to Persuade a Decision Maker?

By Tristan Tomala, Marie Laclau, Frédéric Koessler

Photo Credits: Fergregory / Adobe Stock

Decision Sciences

Black Swans and Other Challenges to Rational Decision Making

By Stefania Minardi, Itzhak Gilboa

viral videos - AdobeStock_Editorial_Use_Only
Information Systems

When Videos Become Viral: Why, How and What Consequences?

By Haris Krijestorac

donald trump and doctors - vignette - PICRYL
Decision Sciences

How Believing in Unsubstantiated Claims Leads to Polarization

By Anne-Sophie Chaxel

black swan on a lake - vignette - Tatiana-AdobeStock

A New Theory in Economics Helps Predict Future Events

By Itzhak Gilboa

Fan Wang Profile
Fan Wang
Ph.D. Student
Subscribe button for Knowledhe@HEC newsletter

Newsletter knowledge

A monthly brief in your email box and 3 issues of the book per year.

follow us

Insights @HECParis School of #Management

Follow Us

Support Research

Our articles are produced thanks to our reader's support