Skip to main content
About HEC About HEC Faculty & Research Faculty & Research Master’s programs Master’s programs MBA Programs MBA Programs PhD Program PhD Program Executive Education Executive Education Summer School Summer School HEC Online HEC Online About HEC Overview Overview Who We Are Who We Are Egalité des chances Egalité des chances Career Center Career Center International International Campus Life Campus Life Stories Stories The HEC Foundation The HEC Foundation Faculty & Research Overview Overview Faculty Directory Faculty Directory Departments Departments Centers Centers Chairs Chairs Knowledge Knowledge Master’s programs Master in
Management Master in
Management
MSc International
Finance MSc International
Finance
Specialized
Masters Specialized
Masters
X-HEC
programs X-HEC
programs
Dual-Degree
programs Dual-Degree
programs
Visiting students Visiting students Certificates Certificates Student Life Student Life
MBA Programs MBA MBA EMBA EMBA TRIUM EMBA TRIUM EMBA PhD Program Overview Overview HEC Difference HEC Difference Program details Program details Research areas Research areas HEC Community HEC Community Placement Placement Job Market Job Market Admissions Admissions Financing Financing Executive Education Executive Masters Executive Masters Executive Certificates Executive Certificates Executive short programs Executive short programs Online Online Executive MBA Executive MBA Train your teams Train your teams Summer School Youth Leadership Initiative Youth Leadership Initiative Summer programs Summer programs Admissions Admissions FAQ FAQ HEC Online Overview Overview Degree Program Degree Program Executive certificates Executive certificates MOOCs MOOCs

Instant

cascad: A new certifying organization to help double-check scientific results

Data Science

While scientific findings need to be assessed by peers and journal referees, the confidentiality of original data often makes the process arduous. An accredited organization launched by Christophe Pérignon (HEC Paris) and colleagues with access to the original research data can now ensure reproducibility of results. This not only promises huge gains in time and effort for researchers but will also shore up trust in scientific results.

Christophe Pérignon on open science, research reproducibility, and cascad, on Knowledge@HEC Insights

The cornerstone of all scientific research is that every scientific result should be replicable. This means that independent researchers should be able to conduct the same study again and find the same results as a first study found. Over recent years across disciplines ranging from business studies to medicine, increasing efforts are being made to replicate research findings.

The cornerstone of all scientific research is that every scientific result should be replicable.

Replication is not possible, however, if the original data cannot be accessed by independent researchers. This is a bigger issue today than ever before with growing awareness of the importance of data protection and privacy and more and more laws that prevent confidential data from being freely shared. For instance, in economics, around 40% of the empirical papers published in the best academic journals use confidential data.

A new agency for certifying computational research

Christophe Pérignon and colleagues have recently launched the Certification Agency for Scientific Code and Data (cascad). The cascad agency is a not‐for‐profit certification agency created by academics with the support of the French National Centre for Scientific Research. It is a trusted third-party that formally checks whether the results presented in a paper can be obtained from the data and computer code of the researchers.

Certification Agency for Scientific Code and Data (cascad)

The cascad agency offers two kinds of certification: one for research based on open data and another one for research based on confidential data. The latter is being done through collaboration with the Centre d’Accès Sécurisé aux Données (CASD), a French public research infrastructure that allows users to access and work with confidential government data under secured conditions. This centre currently provides access to data from the French Statistical Institute and the French Ministries for Finance, Justice, Education, Labor, and Agriculture, as well as Social Security contributions and health data. Data cannot be downloaded, but access is made possible via a virtual machine that allows researchers to remotely access data on a specific piece of hardware that is protected by a fingerprint reader.

The cascad agency offers two kinds of certification: one for research based on open data and another one for research based on confidential data.

The application process to CASD data takes around six months and involves a presentation of the research project before the French Statistical Secrecy Committee. This creates a major roadblock preventing the referees tasked with evaluating research papers from gaining access, as they currently have to go through exactly the same process as the original researchers. “This has been a clear impediment to research reproducibility and we had to come up with a solution”, explains Pérignon. Now, thanks to the cascad-CASD partnership, researchers have the opportunity to signal the reproducibility of their work based on confidential data. 

Introducing the reproducibility reviewer

Researchers will now be able to request a reproducibility certification for a paper when they want to publish it. Then, a “reproducibility reviewer”, who is a full‐time cascad employee specialized in the software used by the author, verifies the results presented in a paper by accessing a CASD virtual machine, which is a clone of the one used by the author. This includes a copy of the source dataset and of the author’s computer code, as well as all software required to run the code. 

Perignon Knowledge HEC insights

 

Having a dedicated team of reproducibility reviewers with the job of verifying research data brings a range of benefits. “We can do it in a few days rather than in a few months and we can do it systematically rather than waiting for someone who is interested or brave enough or patient enough to come along and do this themselves” says Pérignon. When researchers see that a paper has been certified, they know that a third-party has been able to use the very same code and data as the original researchers and has successfully reproduced the results. This boosts trust in the published results and in science.

Enriching the academic review process

Currently it is expected that journals take care of the academic review process. They enlist researchers who volunteer their time to evaluate research. In most cases, they have no access to the core data used to produce the work. Researchers do this to help the community but they are completely swamped by the volume of work they already have. In reality, checking that research papers accurately represent the data they are based on takes special skills and quite a bit of time. So “there are economies of scale of having a specialized agency with full-time experts in software and data. Journals can outsource this activity to a specialized agency like cascad,” explains Pérignon. This frees up the resources of researchers who review work and allows them to have greater confidence in the work they review.

Beyond academia

Now that cascad is up and running, there is no reason it need be limited to academia. “The tool that we designed here can go beyond academia; it could be used to redo analyses made by all sorts of agencies and to test algorithms used by corporations and public administrations”. Given the increasingly important role played by algorithms in our society, having a trusted third-party being able to certify that a given algorithm is valid and unbiased would be extremely useful.
 

Article based on an interview with Christophe Pérignon about "cascad". 

Related content on Data Science

Raphael Levy
Raphaël Levy
Assistant Professor
Data Science
How can data help people's lives?
Hugues Langlois HEC video
Finance

Big Data and Investment Returns: Insights by HEC Professor Hugues Langlois

By Hugues Langlois-Bertrand

En Route For The "Sexiest Job of the 21st Century"?
Data Science

En Route For The “Sexiest Job Of The 21st Century"?

Newsletter knowledge

A monthly brief in your email box and 3 issues of the book per year.

follow us

Insights @HECParis School of #Management

Follow Us

Support Research

Our articles are produced thanks to our reader's support