What is meta-research?
Meta-research has been defined as ‘the study of research itself: its methods, reporting, reproducibility, evaluation and incentives.’ (1) As research on research, meta-research uses scientific methods to explore how research is done and interpreted, with the goal of verifying the validity of research techniques and improving the reliability of conclusions.
(1) Ioannidis JPA. Meta-research: Why research on research matters. PLoS Biol.
2018 Mar 13;16(3):e2005468. doi: 10.1371/journal.pbio.2005468
How do I submit a protocol?
To register a protocol, you need to first sign up as a user of IRMR. Your login details will allow you to update your record if and when required. You then need to submit some basic information and upload the protocol itself. Once submitted, we will screen your submission to check that it fulfils our definition of a protocol for meta-research. When we have confirmed eligibility, the record will be permanently entered on to the register.
What are the focal areas of meta-research?
The Meta-Research Innovation Center at Stanford (METRICS) has divided meta-research in to five focal areas. (1)
When you register your protocol, you will be asked to choose the focal area that best describes your project.
The five focal areas are:
Methods – The phase of designing and conducting research
Rigorous research methods are at the heart of good science. These methods include everything from question formation to study design, conduct, statistical analysis and interpretation. Meta-researchers use both theoretical and empirical approaches to identify flaws and biases in the research enterprise, then develop and test methods to minimize them and define best practices.
Reporting – The phase of communicating research
For research to be useful it needs to be communicated properly to scientists and others who use the results. This reporting is done through research papers, written by scientists, or by various forms of media that convey scientific results to broader audiences. Meta-researchers are developing standards for reporting results and experimenting with new means to improve communication to scientists, policy makers, patients, and the general public.
Evaluation – The phase of evaluating research
Scientific quality is constantly evaluated in journal publications, funding decisions, academic promotions, industry investment, medical practice guidelines, standard setting, and policy decisions. Peer review has long served as the cornerstone of such evaluations, but it is widely acknowledged to be imperfect and inefficient. Meta-researchers across multiple disciplines are assessing its effectiveness and experimenting with new approaches to improve the assessment of scientific quality.
Reproducibility – The phase of verifying research
Replicating research results is central to scientific credibility. Meta-research has repeatedly shown that accurate reporting and sound peer-review do not by themselves guarantee the reproducibility of a scientific claim and that active replications of previous research are too rarely done. Meta-researchers are developing new ways to assess the reproducibility of findings, make experimental data available, and promote policies that can make verifying research more routine and more effective.
Incentives – The phase of rewarding research
Research is a rewarding activity in itself, but funding, recognition, and career advancement are critical to a successful scientific career. If the scientific reward system is not aligned with the adoption of best scientific practices, such as study replication and publishing negative results, such practices will happen rarely. Changing the criteria and processes by which rewards are distributed in science can help ensure that the most reliable research is encouraged.
Will the record be screened?
We will screen the record to confirm that it fulfils our definition of a protocol for meta-research. We will not review the protocol with regard to its scientific quality.
How to make changes to a record
By using your login details, you can make changes to a record at any time. Changes will be logged as amendments and the most recent version will be available for immediate viewing on the register. Previous iterations will also remain present and visible within the record, allowing for full transparency.
How to cite an IRMR registration
We recommend that you cite your IRMR registration in the following way:
Author(s). Title. IRMR: International Register of Meta-Research. Year of registration. Unique Identification number (IRMR_xxxxxx). Available from: hyperlink for the record.
Chong S, The Relationship of Study Findings to Publication Outcome in Anaesthesia Research Post Implementation of Mandatory Trial Registration, IRMR: International Register of Meta-Research. 2019. IRMR_00003. https://www.irmr.org/records/IRMR_000003
We also suggest that you include the prospective registration of your protocol in your final report.
A protocol for this research was prospectively registered on IRMR (International Register of Meta-Research). This registration, including a full copy of the protocol, can be accessed at https://www.irmr.org/records/IRMR_000003.