For detailed information about the preparation of a Cochrane protocol or a review, peer reviewers of Intervention protocols or reviews are referred to the Cochrane Handbook for Systematic Reviews of Interventions  and of Diagnostic Test Analysis protocols or reviews are referred to the Cochrane Handbook for Diagnostic Test Accuracy Reviews (in development).

The aim of a Cochrane review is to assess systematically and thoroughly the best possible scientific evidence about the effects of a health care intervention(s). Everything about the review should aim to minimise the possibility of ending in a conclusion that is biased (systematic error either through domains or through design) or based on play of chance (random errors). This means:

• the conduct of the review and its analyses should follow clear, pre-specified criteria - with checks along the way
• it should be easy to understand
• any conflict of interest of the people doing the review should be declared
• effort must be made to find every possible study that might be eligible for the review
• the studies included in the review's analyses should be ones which have as little bias in them as possible
• outcomes that are important to the consumers of the interventions must be considered - whether or not they have been measured by researchers - to avoid conclusions being based on a narrow picture of relaity
• the analyses should take into account statistical methods that may reduce the risk of random errors
• the final review should follow the pre-specified criteria, addressing all the important issues originally raised, and highlight any issues and gaps in the information that should be addressed by researchers in future

Cochrane protocols and reviews are published in the the Cochrane Library – an electronic journal that is accessible on-line as well as on a DVD. The software used to prepare a review is Review Manager which provides fixed review formats (depending on the review type) and the review content has a tree structure. Peer reviewers receive the protocol or review in a pdf format.

What is required of a peer reviewer?
Prior to publication, a protocol is reviewed by at least one or more persons from each of the following groups: Cochrane review authors working in the area of interest, consumers or representatives of consumers acting as consumer peer reviewers, health care professionals who are not Cochrane review authors, statisticians or methods expert, developing countries representatives. Peer reviewers should not have direct financial or personal conflicts of interest concerning the topic addressed. 

What is the process?
Peer reviewers are sent a copy of the protocol and a checklist for comments (to be returned to the CHBG Managing Editor. We ask peer reviewers to submit courteous and constructive comments on the protocol that identify its weaknesses or fatal flaws, as well as ways of improving it. Peer reviewers are usually requested to return these comments within a specified in the letter time. 

Usually, one member of The CHBG Editorial Team is appointed Contact Editor of the protocol. The Managing Editor, having collated the peer reviewer’s comments, forwards these to the Contact Editor and to the authors.

The revised and final version of the protocol has to be approved by the Contact Editor or The Co-ordinating Editor or both prior to its publication on the Cochrane Library.

Peer reviewers would be helped in commenting on a protocol if they know "What is a protocol"?
Cochrane reviews go through three stages of development. At first, people who want to do a review propose a title, with an outline of what the review would address. They need to get this title accepted and registered by a Cochrane review group before they can start. (The exact wording of the title can be changed at any later stage of the process, though this is better avoided.)

Registration of the title means the review authors can proceed their work on the protocol development, ie, the second review stage. A protocol should set out clearly why the review is needed, what the review is going to be about, and how the review authors are going to do it. The protocol is the vehicle for publicly setting out the pre-specified criteria that the review authors must follow. It is published in the Cochrane Library. People can send in comments on protocols in the same way as they can comment on or criticise full reviews. 

The protocol will need to address the following issues:

1.         The specific purpose (question to be answered) of the review.
2.         The types of participants, intervention, and outcomes.
3.         Explicit criteria for deciding which studies to include in the review.
4.         A reasonably comprehensive search strategy.
5.         Avoidance of bias in the selection of articles.
6.         Criteria for assessing the risk of systematic errors ('bias') of the included randomised clinical trials.
7.         Criteria for assessing the risk of random errors ('play of chance').
8.         Methods (whether qualitative or quantitative) for combining the findings.

A protocol has several sections (outlined below), which are kept as the introductory parts of the final review. 

Title
The title of the review should properly reflect the topic of the review, and it should be easy to understand. It must contain the intervention(s) reviewed as well as the problem addressed.

Background
This should explain the topic being reviewed, and the intervention(s) of interest. The background should make the motivation and rationale for the review clear. The background section is designed to explain to people what is going to be reviewed and why. The review authors must explain why the question asked is important to answer. For example, it should indicate the areas of uncertainty in relation to the intervention and highlight issues that are controversial or the subject of public concern.  All terms and interventions must be clearly defined, and the background must use a balanced tone that does not pre-judge the value of the intervention.

The background should be brief, up to a few pages long. It is not a monograph or an overview and should be concise and clear.

Objective
This should have a precise statement of the primary objective of the review, including the intervention(s) reviewed and the problem addressed. It can also mention why the review is being done, and how it might relate to a wider review of a general problem.

If there are hypotheses for the review (specific theories or suggestions being tested), these should be stated here. The comparisons that are listed later in the review should be consistent with any hypotheses described here.

Selection criteria
This section has several parts. Together, they should make it clear which studies can be included in the review, and which will not be eligible. The aim is to come up with very specific guidelines for deciding whether a study addresses the objective of the review. This section is supposed to make the reasons for including a study so clear, that anyone else could come along, apply the criteria, and come to more or less the same decisions. This section includes:

Type of studies: This specifies the design of the studies that will be eligible. The aim is to include study designs that minimise the chances of the results being biased. Bias is a systematic error that will run through the study, because what is observed may not be the effect of the intervention, but an effect of bias instead. Bias is usually overestimation of benefit or underestimation of harm. An example is when everybody knows they have received a new 'breakthrough' treatment - just knowing that might make people feel better, while the people who 'missed out' might be very dissatisfied and feel worse because of it.

Type of participants: This needs to state which groups of people can be included in any studies. For example, some reviews might be looking only at children or people over a particular age. Or they might be looking only at people with a specific disease, of a particular severity (for example, a particular level of severity of cirrhosis).

Type of interventions: An intervention is anything that is meant to change the course of events for someone. Surgery, a drug, a treatment, a test, counselling, giving someone a pamphlet - all of these are interventions.

Type of outcomes: The outcomes that the review authors are going to look for in each study should be listed here. This section should include all the most important outcomes for patients and physicians that need to be considered to make decisions about the particular intervention. These need to be specified ahead of time. The studies may not address particular outcomes - in which case it is important for the review authors to know that important topics have been overlooked.

If an outcome is an important key to understanding or assessing the impact of an intervention, this needs to be stated in the protocol. Coming up with theories about outcomes later does not have the same scientific strength.

Search methods for the identification of studies
This section shows how and where the review authors will find the trials that could be eligible for the review. 

The write-up of this section should include the databases that will be searched and the other methods that will be used to find trials. It is recommended that several of the main databases are searched as well as those which may be particularly relevant to the topic of the review. Additional searching includes scanning the lists of references of relevant articles and books, checking through the proceedings of conferences that present research results, personal contact with experts or institutions that could have relevant material, and a search for gray literature. Any journals to be handsearched (that is, looking through all the copies of the journals) should also be listed. Cochrane reviews are meant to try and identify both published and unpublished studies. That is because unpublished studies more often have neutral or negative findings. Leaving them out, may bias results ('publication bias').

At least one search strategy, line-by-line, of one major database to be searched, should be included in the Appendix.

Data collection and analysis
This section should spell out the precise steps that will be taken to prepare the review, that is, which review author is going to undertake which task, and according to what standards and criteria. Review authors need to outline who will be deciding what trials will be included in the review, who will be extracting data from included trials, and how discrepancies between review authors involved in these processes will be resolved. It is recommended that at least two review authors independently select trials eligible for the review and independently extract data, with discrepancies resolved by discussion or consultation with a third author.

This section of the protocol should also make it clear how the risk of bias of the included trials is going to be assessed, what the criteria are going to be for assessing bias risk, and what checks there will be on these steps. As with the selection criteria, the aim is to be clear and specific so that someone else applying the same methods would come up with more or less the same results.

The review authors should also include in this section the statistical methods they will use if they are pooling the results of different studies. In order to reduce risk of random errors ('play of chance'), The Cochrane Hepato-Biliary Group advises to control for risks of type I and type II errors by using Trial Sequential Analysis or similar methods. As with outcomes, any additional sensitivity and subgroup analyses need to be specified up front before the review is undertaken. It is more reliable to establish reasons that there might be differences in the data before the analysis is done, and then test these hypotheses, than to imagine reasons for observed differences after the analysis has been undertaken. It is to be noticed that assessment of imprecision with GRADE and ssessment of imprecision with Trial Sequential Analysis shall be compared in Sensitivity analysis. Trial Sequentail Analysis assessment of imprecision result shall not be mixed together with the assessment of imprecsion result following GRADE or reported in the GRADE Summary of Findings table.

Finally, plans for considering consumer views, and any consumer participation, should be outlined within the methods section.

Conflict of interest
Review authors should report here any conflict of interest capable of influencing their judgments, including personal, political, academic, and other possible conflicts, in particular financial conflicts.  It is impossible to abolish all conflict of interest, since the only person who does not have some vested interest in a subject is somebody who knows nothing about it at all, and who cannot be affected in any way.  However, any interest that could unduly influence judgments in a review (such as deciding which studies can stay in, or what the results mean) needs to be declared. 

Financial conflicts of interest in particular need to be declared. This includes the receipt of any benefit in cash or kind, any hospitality, or any subsidy derived from any source with an interest in the results of the review.  Any sponsorship or funding of the review should be declared.

If there are no conflicts of interest, this should be stated explicitly, eg. by reporting ‘None known’. 

Authors must get familiar with and must follow the Conflicts of interest and Cochrane Reviews.

References
References for every article or book identified in the text need to be listed, complying with Cochrane standard format for references and citations (see the Cochrane Style Manual).

Style of writing
The text of the review should be clear and to the point. The average length of a protocol is 5 to 10 pages. It should be written so that someone who is not an expert in the area can understand it.

Peer reviewers would be helped in commenting on a review if they know "What is a systematic review?"
A systematic review attempts to identify, appraise, and synthesise all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimising bias, in order to produce more reliable findings that can be used to inform decision making. (See Section 1.2 in the Cochrane Handbook for Systematic Reviews of Interventions) Researchers conducting systematic reviews use explicit methods aimed at minimising random errors (See Manual for Trial Sequential Analysis).

Cochrane Reviews are systematic reviews of research in healthcare and health policy that are published in the Cochrane Database of Systematic Reviews. There are four types of Cochrane reviews:

1. Intervention reviews assess the benefits and harms of interventions used in healthcare and health policy.
2. Diagnostic test accuracy reviews assess how well a diagnostic test performs in diagnosing and detecting a particular disease.
3. Overview of reviews.
4. Methodology reviews address issues relevant to how systematic reviews and clinical trials are conducted and reported.

Cochrane reviews base their findings on the results of trials which meet certain quality criteria, since the most reliable studies will provide the best evidence for making decisions about health care. Authors of Cochrane reviews apply methods which reduce the impact of bias across different parts of the review process, including:

1. Identification of relevant studies from a number of different sources (including unpublished sources).
2. Selection of studies for inclusion and evaluation of their strengths and limitations on the basis of clear, predefined criteria.
3. Systematic collection of data.
4. Appropriate synthesis of data.

These methods are described in detail in the Cochrane Handbook for Systematic Reviews of Interventions and the Cochrane Handbook for Diagnostic Test Accuracy Reviews (in development).

Cochrane reviews are updated to reflect the findings of new evidence when it becomes available because the results of new studies can change the conclusions of a review. Cochrane Reviews are therefore valuable sources of information for those receiving and providing care, as well as for decision-makers and researchers.

What is a meta-analysis?
If the results of the individual studies are combined to produce an overall statistic, this is usually called a meta-analysis. Many Cochrane Reviews measure benefits and harms by collecting data from more than one trial, and combining them to generate an average result. This aims to provide a more precise estimate of the effects of an intervention and to reduce uncertainty ('precision and power').

Not every review in the Cochrane Database of Systematic Reviews contains a meta-analysis. This might not be appropriate if the designs of the studies are too different, if the outcomes measured are not sufficiently similar, or if there are concerns about the quality of the studies, for an average result across the studies to be meaningful.

The CHBG's policy on commenting protocols and reviews
The peer reviewers’ process is ‘open’, ie, no attempt will be made to hide the identity of the peer reviewers from any one involved in the process as well as the general public. All feedback forms and other comments will be forwarded to the review authors. It is therefore important that no demeaning statements are included and that the comments raised are used to increase the quality.

The authors are requested to respond to the raised comments. The Contact Editor or the Co-ordinating Editor will have the final decision as to which of the peer reviewer’s’ comments need to be addressed in the event of disagreement between a peer reviewer and a team of authors.

- All peer reviewers will be asked to declare any potential conflict of interest.

- All peer reviewers are requested to adhere to the Committee on Publication Ethics (COPE) peer review ethical guidelines.

- ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research