Statistics Registration Checklist
Evaluating the Report of a Preregistered Study
Information about registration
Prospective registration of experiments is very valuable for controversial research areas such as parapsychology. Registration greatly increases the reliability and confidence for reviews or meta-analyses of a line of research. The registered experiments provide a database that is not subject to possible reporting or publication biases. Registration also increases confidence by providing clear evidence that the key hypotheses and analyses were planned prior to conducting the experiment. This documented planning of experiments enhances the methodological quality for an area of research. In medical research, prospective, public registration of experimental studies is increasingly required as a condition for publication and for regulatory compliance. The registrations must be publicly accessible on a study registry before data collection starts. The medical registries specify required information for study registrations and review the registrations to assure completeness and consistency.
The KPU Study Registry is a simple registry that makes the provided information publicly available. It offers researchers, reviewers, editors and others the ability to identify studies that were planned for a line of research and to verify that a study report matches the pre-registered information.
Several other registration options for the behavioral sciences are available and have different strengths and weakness. We have published a discussion of this rapidly changing area, identifying strengths and weaknesses for specific options, and making recommendations to improve registration practices. We have also added comments to the online article to update the discussion with recent developments.
About the KPU Registry
The KPU Study Registry is primarily intended for confirmatory studies in parapsychological research, but exploratory studies can also be registered. Registration should be done prior to testing the first participant, but late registration is preferred to no registration. The information sent in for registration will be placed on this website for public viewing and tracking. Please click here to see the registered studies.
The minimum contents for registration of a study are:
1. The title or name of the study (for listing the study in the registry).
2. The name, affiliation, and email address for the lead researcher(s) for the study.
3. A short description or abstract of the purpose and design of the study.
4. A statement or list of the specific hypothesis or hypotheses being tested, and whether each hypothesis is confirmatory or exploratory. A discussion of confirmatory and exploratory analyses is provided here.
5. The planned number of participants and the number of trials per participant.
6. A statement that the registration is submitted prior to collecting data for the first participant, or indicating the number of participants that have provided data when the registration (or revision to the registration) was submitted.
The following additional information is needed for studies that include confirmatory analyses:
7. Specification of all analysis decisions that could affect the confirmatory results, including: the specific statistical test for each confirmatory hypothesis, whether the test is one-sided or two-sided, the criterion for acceptable evidence, any transformations or adjustments to the data, any criteria for excluding or deleting data, and any corrections for multiple analyses. Checklists and examples for registering different types of analyses are provided in the statistics registration document.
8. The power analysis or other justification for the number of participants and trials.
9. The methods for randomization in the study.
10. A detailed description of the study procedure.
This template can be used to provide study information. The registration information (in English, please) should be submitted in Word format and sent as an attachment to an email to Caroline Watt. After the submitted document has been reviewed for consistency with registration requirements, Professor Watt will add to the Word file the registry ID number and date received, and then will save the file as a PDF whose filename includes the ID number. A registration can be revised, but the initial and revised version will both be posted if the revision is after the first participant has been tested.
The KPU Registry will also post information about the results of a registered study if the researcher provides that information. We encourage researchers to provide the abstract and reference when a registered study is published. Researchers may also provide a summary of the results of a registered study prior to publication. The information should be emailed to Caroline Watt as a document in Word format or as a URL to another website.
Study preregistration has become more widely used in recent years. When properly done, it is a valuable tool for preventing researcher bias, however less attention has been paid to its potential as a powerful tool for identifying bias in the report of a preregistered study. This document recommends how to use the preregistration when evaluating the subsequent report of a preregistered confirmatory study. The evaluation may be part of peer review for publication, or an evaluation of the credibility of a study in a systematic review of a line of research.
This study registry is operated by Caroline Watt and Jim Kennedy following these Standard Operating Procedures.