Information for authors

The below text is part of The CHBG Module, published in every issue of The Cochrane Library. The text is usually updated after discussions with CHBG editors and other people involved in the review preparation, or whenever the Cochrane Editorial and Methods Department updates methodological requirements.

Depending on the type of review you are working on, you should follow, in general, the guidelines in the Cochrane Handbooks, i.e. The Cochrane Handbook for Systematic Reviews of Interventions or The Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Authors shall prepare and submit their protocols and reviews only after ensuring that all co-authors have approved of it and that the style of the respective protocol or review is consistent with the Cochrane Style Manual. Please remember to use active voice where sensible. In a sentence written in the active voice, the subject of the sentence performs the action [We will include only randomised clinical trials.]. In a sentence written in passive voice, the subject receives the action [Randomised clinical trials were included.].

In addition, authors of intervention reviews shall consider following CHBG guidelines dealing with important methodological and statistical issues, and for which usage, consensus with CHBG editors is achieved.

Methods used in reviews

Outcomes
The CHBG works on standardisation of hepato-biliary outcomes in CHBG review protocols based on the disease condition reviewed. We do already have a standardised set of outcomes for hepatitis B and C. Suggestions for standardised outcomes in other diseases are most welcome.

In general, selection of outcomes in review protocols and their listing shall follow the Guidelines of The Cochrane Handbook for Systematic Reviews of Interventions. In the Handbook, on p.88 to p.90 you will read:

5.4.2 Prioritizing outcomes: main, primary and secondary outcomes
 
Main outcomes
Once a full list of relevant outcomes is compiled for a review, authors should select the main outcomes relevant to the review question. Main outcomes are those outcomes that are essential for the decision-making and shall form the basis of the 'Summary of findings' tables.

'Summary of findings' tables provide key information about the amount of evidence for important comparisons and outcomes, the quality of the evidence, and the magnitude of effect (See Handbook Chapter 11, Section 11.5). There should be no more than seven main outcomes which should generally not include surrogate or interim outcomes. The outcomes should not be chosen on the basis of any anticipated or observed magnitude of effect or chosen because the outcomes are likely to have been addressed in the studies to be reviewed. 

Primary outcomes
Primary outcomes for the review should be identified among the main outcomes, referred above. Primary outcomes are those outcomes that are expected to be analysed should the review identify relevant studies, and conclusions about the effects of the interventions under review will be based largely on these outcomes. There should in general be no more than three primary outcomes, and the primary outcomes should consist of at least one desirable and at least one undesirable outcome (to assess beneficial and adverse effects, respectively).

Secondary outcomes
Main outcomes not selected as primary outcomes would be expected to be listed as secondary outcomes. In addition, secondary outcomes may include a limited number of additional outcomes that the review intends to address. These may be specific to only some comparisons in the review.

For example, laboratory tests and other surrogate measures may not be considered as main outcomes as they are less important than clinical endpoints in informing decisions, but they may be helpful in explaining effect or determining intervention integrity (see Chapter 7, Section 7.3.4).

Box 5.4.a summarizes the principal factors to consider when developing criteria for the 'Types of outcomes'.'' (end of citation)

2. Review protocol outcomes should include clinical outcomes, and review authors shall not be misguided by the clinical outcomes reported in the trials identified for inclusion in the review. Trial culture shall never be the culture of systematic reviews, as most trialists, for example, select ten to fifteen outcomes but report only on a selected few.

3. All-cause mortality should stand alone, and it should be the first primary outcome.

4. Morbidity from the disease could be the second primary outcome.

5. Adverse events should be included as a primary outcome unless the review topic or title formulation precludes the occurrence of an adverse event.

6. Quality of life, even that it is seldom reported, should be included as a primary outcome or as one of the secondary outcomes.

7. Surrogate outcomes (especially non-validated ones) should be included only as secondary or exploratory outcomes. (Exploratory outcomes subheading is missing in RevMan, so please add.)

8. Health economics. This outcome should preferably be the subject of a separate review, see Chapter 15 in the Handbook.

9. Composite outcomes. If trial authors have failed in reporting the separate components of composite outcomes in separate, it is up to the judgement of the review authors if they would meta-analyse them together or not."

The CHBG continues working on defining core outcome sets in CHBG reviews. We have already defined the outcomes for chronic hepatitis B and chronic hepatitis C virus infections. In addition to getting better understanding of the outcomes in Cochrane reviews by patients, physicians, and other users, authors will also be helped in the preparation of overview of reviews and when designing 'Summary of findings' tables in the intervention reviews, as data for the same meaningful outcomes are expected to be found across reviews.

Study selection
The CHBG recommends inclusion of randomised clinical trials for assessment of benefits and harms of interventions. As adverse events may not be caught in small or even large randomised clinical trials, The CHBG encourages also the inclusion of quasi-randomised studies, cohort studies, and case-control studies when dealing with reports of harmful effects of interventions. However, evidence on harm from non-randomised studies shall not be combined with evidence on harms from randomised trials in meta-analyses. The CHBG does not recommend extensive searches for non-randomised studies because our knowledge on how to do this best is limited. However, we appeal to review authors to consider adverse events from both randomised clinical trials and non-randomised studies, the latter usually identified through the searchers for randomised trials.

Authors must follow the guidelines in Chapter 14 of the The Cochrane Handbook for Systematic Reviews of Interventions about adverse events. Two authors should generally perform the selection of studies and data extraction independently. Therefore, the Editorial Team encourages at least two authors to work on a systematic review.

Assessment of risk of bias in randomised trials
The bias risks of the randomised trials included in the reviews is assessed separately and independently by authors of the review using the assessment criteria defined in the protocol. This should follow the The Cochrane Handbook for Systematic Reviews of Interventions. Eventual differences in the bias risk of trials are resolved by discussion in order to reach consensus.

Methodological studies indicate that trials with unclear or inadequate methodological quality may be associated with risk of bias (systematic error) when compared to trials using adequate methodology.1-15 Such bias may lead to overestimation of intervention benefits and underestimation of harms.

There is evidence that trials with adequate randomisation (both sequence generation and allocation concealment), blinding, and follow-up generate the most valid results. Unfortunately, such trials are often not available for meta-analyses. Of 370 drug trials, 28% reported adequate generation of the allocation sequence, 22% reported adequate allocation concealment, and 63% were double blind.7 Accordingly, only 4% were adequate regarding all components.7 Subgroup analyses and meta-regression analyses are, therefore, important to evaluate the influence of risk of bias on the results.

Based on the recommendations in the The Cochrane Handbook for Systematic Reviews of Interventions and methodological studies2-4;6;8;10;12-15, we suggest that authors of systematic reviews use the below standard text and definitions in the assessment of bias risk in a trial. Please note that specific circumstances may sometimes necessitate changes in the text or definitions, or the use of additional risk of bias domains.

We suggest that authors perform overall assessment of the bias risk of trials irrespective of outcome as well as according to outcome. The latter can be displayed in Summary of Findings tables (See Handbook Chapter 11: Presenting results and ‘Summary of findings’ tables and Chapter 12: Interpreting results and drawing conclusions).

"Assessment of risk of bias in included studies

Two review authors (XX and YY) in pairs will independently assess the risk of bias in the included studies. We will assess risk of bias according to the Cochrane risk of bias tool (Higgins 2011), the Cochrane Hepato-Biliary Group Module (Gluud 2018), and methodological studies (Schulz 1995; Moher 1998; Kjaergard 2001; Wood 2008; Savović 2012a; Savović 2012b; Lundh 2017; Savović 2018), using the following sources of bias, defined as follows:

Allocation sequence generation

  • Low risk of bias: sequence generation was achieved using computer random number generation or a random number table. Drawing lots, tossing a coin, shuffling cards, and throwing dice awere adequate if performed by an independent person not otherwise involved in the trial.
  • Uncertain risk of bias: the method of sequence generation was not specified.
  • High risk of bias: the sequence generation method was not random.

Allocation concealment

  • Low risk of bias: the participant allocations could not have been foreseen in advance of, or during enrolment. Allocation was controlled by a central and independent randomisation unit; or the allocation sequence was unknown to the investigators (for example, if the allocation sequence was hidden in sequentially numbered, opaque, and sealed envelopes).
  • Uncertain risk of bias: the method used to conceal the allocation was not described so that intervention allocations may have been foreseen in advance of, or during enrolment.
  • High risk of bias: the allocation sequence was likely to be known to the investigators who assigned the participants.

Blinding of participants and personnel

  • Low risk of bias: any of the following: blinding of participants and key study personnel ensured, and it was unlikely that the blinding could have been broken; or rarely no blinding or incomplete blinding, but the review authors judged that the outcome was not likely to be influenced by lack of blinding.* (*Great caution is needed if deciding on making this judgement. Therefore, it is up to the authors's judgement to decide at the protocol level what hard outcomes they consider as outcomes not influenced by lack of blinding or incomplete blinding of participants and personnel).
  • Unclear risk of bias: any of the following: insufficient information to permit judgement of 'low risk' or 'high risk;' or the trial did not address this outcome.
  • High risk of bias: any of the following: no blinding or incomplete blinding, and the outcome was likely to be influenced by lack of blinding; or blinding of key study participants and personnel attempted, but likely that the blinding could have been broken, and the outcome was likely to be influenced by lack of blinding.

Blinded outcome assessment

  • Low risk of bias: any of the following: blinding of outcome assessment ensured, and unlikely that the blinding could have been broken; or rarely no blinding of outcome assessment, but the review authors judged that the outcome measurement was not likely to be influenced by lack of blinding.* (*Great caution is needed if deciding on making this judgement. Therefore, it is up to the authors's judgement to decide at the protocol level what hard outcomes they consider as outcomes not influenced by lack of blinding or incomplete blinding of outcome assessors).
  • Unclear risk of bias: any of the following: insufficient information to permit judgement of 'low risk' or 'high risk;' or the trial did not address this outcome.
  • High risk of bias: any of the following: no blinding of outcome assessment, and the outcome measurement was likely to be influenced by lack of blinding; or blinding of outcome assessment, but likely that the blinding could have been broken, and the outcome measurement was likely to be influenced by lack of blinding.

Incomplete outcome data

  • Low risk of bias: missing data were unlikely to make treatment effects depart from plausible values. The study used sufficient methods, such as multiple imputation, to handle missing data.
  • Unclear risk of bias: there was insufficient information to assess whether missing data in combination with the method used to handle missing data were likely to induce bias on the results.
  • High risk of bias: the results were likely to be biased due to missing data.

Selective outcome reporting

  • Low risk of bias: the trial reported the following predefined outcomes: mortality, serious adverse events, and XXX morbidity (as examples). If the original trial protocol was available, the outcomes should have been those called for in that protocol. If the trial protocol was obtained from a trial registry (e.g. www.ClinicalTrials.gov), the outcomes sought should have been those enumerated in the original protocol if the trial protocol was registered before or at the time that the trial was begun. If the trial protocol was registered after the trial was begun, we will not consider those outcomes to be reliable.
  • Unclear risk of bias: the study authors did not report all predefined outcomes fully, or it was unclear whether the study authors recorded data on these outcomes or not.
  • High risk of bias: the study authors did not report one or more predefined outcomes.

For-profit bias

  • Low risk of bias: the trial appeared free of industry sponsorship or other type of for-profit support that could manipulate the trial design, conductance, or trial (industry-sponsored trials overestimate the efficacy by about 25%) (Lundh 2017).
  • Unclear risk of bias: the trial may or may not have been free of for-profit bias as the trial did not provide any information on clinical trial support or sponsorship.
  • High risk of bias: the trial was sponsored by industry or received another type of for-profit support.

Other bias*

  • Low risk of bias: the trial appeared free of other factors that could put it at risk of bias.
  • Unclear risk of bias: the trial may or may not have been free of other factors that could put it at risk of bias.
  • High risk of bias: there were other factors in the trial that could put it at risk of bias.

*Authors should think what other bias in addition to the above defined biases may be relevant for their review, and if other bias specific to their review question is identified, then authors should report on it, adapting the text in the above pattern.

Overall risk of bias

We will assess overall risk of bias in the trials as:

  • low risk of bias: if all the bias domains described in the above paragraphs are classified as low risk of bias;
  • high risk of bias: if one or more of the bias domains described in the above paragraphs are classified as 'unclear risk of bias' or 'high risk of bias.'

We will assess the domains 'Blinding of outcome assessment,' 'Incomplete outcome data,' and 'Selective outcome reporting' for each outcome. Thus, we will be able to assess the bias risk for each outcome in addition to each trial.

We will base our primary conclusions and our presentation in the 'Summary of findings' table on the results of our primary outcomes at low risk of bias."

Authors should also consider design issues, eg, the administration of inappropriate treatment being given to the controls such as suboptimal dosage of medication or a supraoptimal dosage of medication that may bias a comparison.

The domains 'baseline imbalance' and 'early stopping of trials' shall not be routinely judged when assessing the risk of bias in an included trial of a systematic review. The argumentation for not considering baseline imbalance is that this imbalance may occur due to random error ('play of chance'), and that such a random error is likely to be levelled out by conducting a meta-analysis of several trials. The argumentation for not considering early stopping is that such trials - although they are likely to overestimate intervention effects - are counterbalanced by trials finding no significant difference.

Trials assessed as having 'low risk of bias' in all of the specified in the review individual domains shall usually be considered 'trials with low risk of bias'1-14. Trials assessed as having 'unclear risk of bias' or 'high risk of bias' in one or more of the specified in the review individual domains shall be considered trials with 'high risk of bias'1-14.

In a large number of reviews, such optimal division of trials may not be possible, simply due to the fact that there are no or there are very few trials with low risk of bias. If review authors have a suspicion that this may be so, they should try to formulate alternative ways of defining trials with 'lower risk of bias' based on fewer domains. Such definitions should preferably be considered at the protocol stage, that is, well before embarking on data extraction and analyses. However, when drawing conclusions, it has to be remembered that no or only few trials with low risk of bias existed. Hence, the chance to know the 'true' intervention effect is low or absent.

Data collection
Generally, two or more authors should extract data independently regarding inclusion criteria (design, participants, interventions, and outcomes), criteria for risk of bias, and results. When data are missing in a published report, authors should contact the corresponding author of the trial report. Collection of data from unpublished studies must be performed by writing to authors of previously published studies as well as the industry or manufacturers of the intervention. Any substantial piece of information regarding unpublished data should be entered as a reference. For the correct type of the reference, please see The Cochrane Style Guide.

Analysis
Statistical methods of RevMan Analyses are used for analysing the data. All analyses should include an analysis according to the intention-to-treat method. We urge authors of systematic reviewers to follow the instructions in The Cochrane Handbook for Systematic Reviews of Interventions regarding statistical analyses. Sensitivity analyses may be performed. Furthermore, the short instructions below can assist in writing the statistical methods section in your review.

How to write the 'Statistical methods' section in Cochrane reviews on interventions

Before you start writing the 'statistical methods' section in a protocol for a Cochrane review, you need to consider thoroughly which methods would be most appropriate with regard to your specific question. You should consult The Cochrane Handbook1 where you will find a thorough presentation of most of the statistical methods used in meta-analysis. Overall, the writing of 'statistical methods' in a review is not fixed and should be changed according to the need and characteristics of every unique systematic review. Below, you will find a very brief introduction on how to prepare the 'statistical methods' section including some examples. You need to specify the main software used in the review. This is of usually The Review Manager (RevMan): 'We will use the software package RevMan 5 provided by The Cochrane Collaboration (Review Manager (RevMan) [Computer program]. Version 5.2. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2012.).' Any additional software could also be mentioned here. You should specify the summary statistics for the kind of data you plan to analyse in your review (eg, relative risk for dichotomous data and mean difference for continuous data). The CHBG recommends to apply both a fixed- and a random-effects model meta-analyses. In case of discrepancies, both results are reported, otherwise only one of the results is reported. An example of wording could be:

'For dichotomous variables, we will calculate the relative risks with 95% confidence interval. We will use a random-effects model15 and a fixed-effect model16 meta-analyses. In case of discrepancy between the two models (eg, one giving a significant intervention effect, the other no significant intervention effect) we will report both results; otherwise, we will report only the results from one of the meta-analyses models.'

Heterogeneity between trials should always be explored by considering the bias risk of trials including domains (see above) and design, clinical setting, patients involved, the interventions, etc. Subgroup analyses, sensitivity analyses, or meta-regression may be appropriate. It is important to define the subgroup analyses at the protocol stage and follow them in the review stage. (If you need to do post hoc subgroup analyses, you should specify the reason sufficiently in the review and interpret the results with great caution.) An example of wording:

'The chi-squared test for heterogeneity was used to provide an indication of between-trial heterogeneity. In addition, the degree of heterogeneity observed in the results was quantified using the I-squared statistic17, which can be interpreted as the percentage of variation observed between the trials attributable to between-trial differences rather than sampling error (chance). We will perform a subgroup analysis in order to compare the intervention effect in trials with low risk of bias (see above) to that of trials with unclear or high risk of bias (ie, trials that lack one or more adequate domain).2-4,10

It is difficult to handle trials with missing data (drop-outs/withdrawals).19 We recommend that you always seek to perform intention-to-treat analysis.

An example of wording for "Dealing with missing data

We will perform an intention-to-treat analysis whenever possible. Otherwise, we will use the data that are available to us (e.g. a trial may have reported only per-protocol analysis results). As 'per-protocol' analyses may be biased, we plan to conduct best-worst case scenario analyses (good outcome in intervention group and bad outcome in control group) and worst-best case scenario analyses (bad outcome in intervention group and good outcome in control group) as sensitivity analyses whenever possible.

For continuous outcomes, we plan to impute the standard deviation from P values according to guidance given in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). If the data are likely to be normally distributed, we plan to use the median for meta-analysis when the mean is not available. If it is not possible to calculate the standard deviation from the P value or the Confidence Intervals, we plan to impute the standard deviation using the largest standard deviation in other trials for that outcome. This form of imputation may decrease the weight of the study for calculation of mean differences and may bias the effect estimate to no effect for calculation of standardised mean differences (Higgins 2011)."

You can include missing data by considering them as treatment failures or treatment successes. Furthermore, you could do extreme case analyses where you consider the drop-outs as failures or successes in the experimental group and as successes or failures in the control group. You need to consider what would be the most appropriate assumption for your specific review.

An example of wording of each of the situations mentioned above is:

Intention-to-treat analyses
Regarding the primary outcomes, we will include patients with incomplete or missing data in sensitivity analyses by imputing them according to the following scenarios.19

- Poor outcome analysis: assuming that drop-outs/participants lost from both the experimental and the control arms experienced the outcome, including all randomised participants in the denominator.
- Good outcome analysis: assuming that none of the drop-outs/participants lost from the experimental and the control arms experienced the outcome, including all randomised participants in the denominator.
- Extreme case analysis that favours the experimental intervention ('best-worse' case scenario: none of the drop-outs/participants lost from the experimental arm, but all of the drop-outs/participants lost from the control arm experienced the outcome, including all randomised participants in the denominator.
- Extreme case analysis that favours the control ('worst-best' case scenario): all drop-outs/participants lost from the experimental arm, but none from the control arm experienced the outcome, including all randomised participants in the denominator.

Per protocol analyses 
Interpretation of per protocol analyses should be cautious as they may be biased.

Cross-over trials
We recommend to those who want to include cross-over trials in their systematic reviews to consider using the analytical methods described by Elbourne 2002 as well as The Cochrane Handbook.1

Visual inspection and analysis of bias
Publication bias and other biases can be explored by visual estimation of funnel plots and different statistical methods. The results of these methods vary with the magnitude of the treatment effect, the distribution of trial size, and whether a one- or two-tailed test is used.24 Therefore, several methods should be explored. We can briefly describe the plans as follows:

"Funnel plot of the primary outcome will be used to provide a visual assessment of whether treatment estimates are associated with study size. We will use two tests to assess funnel plot asymmetry, adjusted rank correlation test,23 and regression asymmetry test.24"

Risks of random errors
When few and small trials are combined in meta-analyses, the risk of introducing random errors increase due to sparse data and due to multiplicity when conducting cumulative meta-analyses with repeating analyses of the same data.28;29 The CHBG, therefore, advises review authors to employ trial sequential analyses of their important meta-analyses 28-34 since the CHBG Editorial Team Group meeting in Copenhagen in April 2009. 

An example of a text in a protocol can be:  

Meta-analysis

We will conduct the systematic review according to recommendations stated in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011) and; according to the eight-step procedure for validation of meta-analytic results in systematic reviews as suggested by Jakobsen and colleagues (Jakobsen 2014). We will meta-analyse data using the statistical software Review Manager 5.3 (RevMan 2014).

'Trial Sequential Analysis

Cumulative meta-analyses are at risk of producing random errors due to sparse data and multiple testing of accumulating data (Brok 2008; Brok 2009; Higgins 2011; Pogue 1997; Thorlund 2009; Wetterslev 2008); therefore, Trial Sequential Analysis (CTU 2011) can be applied to control this risk (Thorlund 2011). The required information size (that is the number of participants needed in a meta-analysis to detect or reject a certain intervention effect) can be calculated in order to control random errors (Wetterslev 2008; Wetterslev 2009). The required information size takes into account the event proportion in the control group, the assumption of a plausible relative risk (RR) reduction, and the heterogeneity of the meta-analysis (Turner 2013; Wetterslev 2008; Wetterslev 2009). Trial Sequential Analysis enables testing for significance to be conducted each time a new trial is included in the meta-analysis. On the basis of the required information size, trial sequential monitoring boundaries can be constructed. This enables one to determine the statistical inference concerning cumulative meta-analysis that has not yet reached the required information size (Wetterslev 2008).

If the trial sequential monitoring boundary is crossed before reaching the calculated information size, we may conclude that sufficient evidence is collected to validly assess benefit or harm, and that inclusion of additional trial data may be redundant. In contrast, if the boundaries for benefit or harm are not crossed, we may conclude that further trials are necessary before a certain intervention effect can be evaluated. Trial Sequential Analysis also allows for assessment of the sufficiency of evidence for a postulated intervention effect. A lack of effect is evident if the cumulative Z-score crosses the trial sequential monitoring boundaries for futility.

We will make relatively conservative estimations of the anticipated intervention effect to control the risks of random error (Jakobsen 2014). Large anticipated intervention effects lead to small required information sizes, and the thresholds for significance will be less strict after the information size has been reached (Jakobsen 2014).

We will analyse all primary and secondary outcomes using Trial Sequential Analysis. These analyses will allow us to calculate the Trial Sequential Analysis-adjusted CIs based on the following assumptions:

Primary outcomes

We will estimate the diversity-adjusted required information size (Wetterslev 2009) based on the proportion of patients with an outcome in the control group. We will use an alpha of XX%, a beta of XX%, and the diversity suggested by the trials in the meta-analysis (Jakobsen 2014).

As anticipated intervention effects for the primary outcomes in the Trial Sequential Analysis we will use the following:

  • Serious adverse events: a relative risk reduction of XX% and the observed proportion of serious adverse events in the control group.
  • All-cause mortality: a relative risk reduction of XX% and the observed incidence of mortality in the control group.
Secondary outcomes

We will estimate the diversity-adjusted required information size (Wetterslev 2009) based on the proportion of patients with an outcome in the control group when analysing dichotomous outcomes, and we will use the observed SD when analysing continuous outcomes. We will use an alpha of XX%, a beta of XX%, and the diversity suggested by the trials in the meta-analysis (Jakobsen 2014).

As anticipated intervention effects for the secondary outcomes in the Trial Sequential Analysis, we will use the following relative risk reductions or increases:

  • Quality of life: observed SD divided by 2.
  • Non-serious adverse events: a relative risk reduction of XX%
  • No serological response: a relative risk reduction of XX% and the observed proportion of participants with no serological response in the control group.
Assessment of significance

We will assess intervention effects with both random-effects (DerSimonian 1986) and fixed-effect meta-analyses (DeMets 1987). We will assess significance using the more conservative point estimate of the two, comprised by the estimate closest to zero effect (Jakobsen 2014). If the two estimates are comparable, we will use the estimate with the widest confidence interval. For analysis of two primary outcomes, we will consider significant a P value less than P < XX (Jakobsen 2014), as this will secure a family wise error rate (FWER) below 0.05. We will apply an eight-step procedure to assess if the results from the meta-analyses have passed the thresholds for significance (Jakobsen 2014)."

Example of a Summary of findings’ table text

"We will create 'Summary of findings' tables for all outcomes reported in the review using GRADE Interactive software (GRADEpro GDT [Computer program]. Version accessed January 2018. Hamilton (ON): McMaster University (developed by Evidence Prime, Inc.), 2015. Available at www.gradepro.org). The GRADE approach appraises the quality of a body of evidence based on the extent to which one can be confident that an estimate of effect or association reflects the item being assessed. The quality of a body of evidence considers within-study risk of bias (methodological quality), indirectness of the evidence (population, intervention, control, outcomes)unexplained heterogeneity or inconsistency of results (including problems with subgroup analyses); imprecision of effect estimates (wide Confidence Intervals (CIs), and a high probability of of publication bias.35-46 We will define the levels of evidence as 'high', 'moderate', 'low', or 'very low'.

These grades are defined as follows.
- High certainty: this research provides a very good indication of the likely effect; the likelihood that the effect will be substantially different is low.

- Moderate certainty: this research provides a good indication of the likely effect; the likelihood that the effect will be substantially different is moderate. 
- Low certainty: this research provides some indication of the likely effect; however, the likelihood that it will be substantially different is high. 
- Very low certainty: this research does not provide a reliable indication of the likely effect; the likelihood that the effect will be substantially different is very high."

Reporting of reviews
For policies on the reporting of reviews (for example on the discussion of results, the use of tables and figures, and the naming of studies), authors must follow the recommendations of The Cochrane Handbook for Systematic Reviews of Interventions.

The Cochrane Collaboration's online learning page for authors is a good source of information and developing skills.

References
1. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2009. Available from www.cochrane-handbook.org.
2. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408-12.
3. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998;352:609-13.
4. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine 2001;135:982-9.
5. Jüni P, Altman D, Egger M. Assessing the quality of controlled clinical trials. BMJ (Clinical Research Ed.) 2001;323:42-6.
6. Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technology Assessment 2003;7:1-76.
7. Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events. JAMA 2003;290:921-8.
8. Kunz R, Vist G, Oxman AD. Randomisation to protect against selection bias in healthcare trials. The Cochrane Database of Methodology Reviews 2002, Issue 4. Art. No.: MR000012. DOI: 10.1002/14651858.MR000012.
9. Kjaergard LL, Liu JP, Als-Nielsen B, Gluud C. Artificial and bioartificial support systems for acute and acute-on-chronic liver failure: a systematic review. JAMA 2003;289:217-22.
10. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ (Clinical Research Ed.) 2008;336(7644):601-5.
11. Gluud LL, Thorlund K, Gluud C, Woods L, Harris R, Sterne JA. Correction: reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine 2008;149(3):219.
12. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J et.al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Annals of Internal Medicine 2012;157:429-38.
13. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, A Ioannidis JP, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JA. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Health Technology Assessment 2012;16(35):1-82.
14. Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L. Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews 2012, Issue 12. Art. No.: MR000033. DOI: 10.1002/14651858.MR000033.pub2.
15. Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L. Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews 2017, Issue 2. Art. No.: MR000033 DOI: 10.1002/14651858.MR000033.pub3.
16. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials 1986;7(3):177-88.
17. DeMets DL. Methods of combining randomized clinical trials: strengths and limitations. Statistics in Medicine 1987;6(3):341-50.
18. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ (Clinical Research Ed.) 2003;327:557-60.
19. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ (Clinical Research Ed.) 1999;319:670-4.
20. Elbourne DR, Altman DG, Higgins JP, Curtin F, Worthington HV, Vail A. Meta-analyses involving cross-over trials: methodological issues. International Journal of Epidemiology 2002;31:140-9.
21. Stedman MR, Curtin F, Elbourne DR, Kesselheim AS, Brookhart MA. Meta-analyses involving cross-over trials: methodological issues. International Journal of Epidemiology. 2011; 40(6): 1732-4.
22. Macaskill P, Walter SD, Irwig L. A comparison of methods to detect publication bias in meta-analysis. Statistics in Medicine 2001:20:641-54.
23. Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics 1994;50:(4):1088-101.
24. Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple graphical test. BMJ (Clinical Research Ed.) 1997;315(7109):629-34.
25. Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technology Assessment 2003;7:1-76.
26. Kjaergard LL, Liu JP, Als-Nielsen B, Gluud C. Artificial and bioartificial support systems for acute and acute-on-chronic liver failure: a systematic review. JAMA 2003;289:217-22.
27. Gluud LL, Thorlund K, Gluud C, Woods L, Harris R, Sterne JA. Correction: reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine 2008;149(3):219.
28. Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. Journal of Clinical Epidemiology 2008;61:64-75.
29. Brok J, Thorlund K, Gluud C, Wetterslev J. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analyses. Journal of Clinical Epidemiology. 2008. DOI:10.1016/j.jclinepi.2007.10.007.
30. Brok J, Thorlund K, Wetterslev J, Gluud C. Apparently conclusive meta-analyses may be inconclusive - Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses. International Journal of Epidemiology. 2009;38(1):287-98.
31. Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JP, Thabane L, Gluud LL, Als-Nielsen B, Gluud C. Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses? International Journal of Epidemiology. 2009;38(1):276-86.
32. Wetterslev J, Thorlund K, Brok J, Gluud C. Estimating required information size by quantifying diversity in random-effects model meta-analyses. BMC Medical Research Methodology 2009;9:86.
33. Thorlund K, Engstrøm J, Wetterslev J, Brok J, Imberger G, Gluud C. User manual for trial sequential analysis (TSA). Copenhagen Trial Unit, Centre for Clinical Intervention Research, Copenhagen, Denmark. 2011;1-115. Available from www.ctu.dk/tsa.
34. Jakobsen J, Wetterslev J, Winkel P, Lange T, Gluud C. Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods. BMC Medical Research Methodology 2014;14:120.
35. Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. Journal of Clinical Epidemiology 2011;64(4):401-6. [ PubMed: 21208779]
36.  Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. Journal of Clinical Epidemiology 2011;64(4):383-94. [ PubMed: 21195583]
37. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. Journal of Clinical Epidemiology 2011;64(4):395-400. [ PubMed: 21194891]
38. Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias). Journal of Clinical Epidemiology 2011;64(4):407-15. [ PubMed: 21247734]
39. Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence--publication bias. Journal of Clinical Epidemiology 2011;64(12):1277-82. [ PubMed: 21802904]
40. Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, et al. GRADE guidelines 6. Rating the quality of evidence--imprecision. Journal of Clinical Epidemiology 2011;64(12):1283-93. [ PubMed: 21839614]
41. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 7. Rating the quality of evidence--inconsistency. Journal of Clinical Epidemiology 2011;64(12):1294-302. [ PubMed: 21803546]
42. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidence--indirectness. Journal of Clinical Epidemiology 2011;64(12):1303-10. [ PubMed: 21802903]
43. Guyatt GH, Oxman AD, Sultan S, Glasziou P, Akl EA, Alonso-Coello P, et al. GRADE guidelines: 9. Rating up the quality of evidence. Journal of Clinical Epidemiology 2011;64(12):1311-6. [ PubMed: 21802902]
44. Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, et al. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. Journal of Clinical Epidemiology 2013;66(2):151-7. [ PubMed: 22542023]
45. Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, et al. GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes. Journal of Clinical Epidemiology 2013;66(2):158-72. [ PubMed: 22609141]
44. Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, et al. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. Journal of Clinical Epidemiology 2013;66(2):173-83. [ PubMed: 23116689]
46. Mustafa RA, Santesso N, Brozek J, Akl EA, Walter SD, Norman G, et al. The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses. Journal of Clinical Epidemiology 2013;66(7):736-42; quiz 742.e1-5. [ PubMed: 23623694]
47. Jakobsen JC, Wetterslev J, Winkel P, Lange T, Gluud C. Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods. BMC Medical Research Methodology 2014;14(1):120.
48. Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ (Clinical Research Ed.) 2012;344:e1119.
49. Hrobjartsson A, Thomsen ASS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ : Canadian Medical Association Journal 2013;185(4):E201-11.
50. (Hrobjartsson 2014a) Hrobjartsson A, Emanuelsson F, Skou Thomsen AS, Hilden J, Brorson S. Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. International Journal of Epidemiology 2014;43(4):1272-83.
51. (Hrobjartsson 2014b) Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Rasmussen JV, Hilden J, et al. Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. International Journal of Epidemiology 2014;43(3):937-48.
52.
Gluud C, Nikolova D, Klingenberg SL. Cochrane Hepato-Biliary Group. About Cochrane (Cochrane Review Groups (CRGs)) 2018, Issue (changes - see The CLib issue). Art. No.: LIVER.
53.
Savović J, Turner RM, Mawdsley D, Jones HE, Beynon R, Higgins JPT, et al. Association between risk-of-bias assessments and results of randomized trials in cochrane reviews: The ROBES meta-epidemiologic study. Am J Epidemiol 2018;187(5):1113-22.

Core outcomes for chronic hepatitis B (CHB) virus infection

Primary outcomes

  • All-cause mortality or hepatitis B-related morbidity (number of participants who developed cirrhosis, ascites, variceal bleeding, hepato-renal syndrome, hepatocellular carcinoma, or hepatic encephalopathy and who have not died). These outcomes will be tested as a composite outcome as well as individually (mortality or morbidity). Such composite outcomes need to be interpreted with caution, especially if the components are influenced differently by the intervention.
  • Health-related quality of life (any valid assessment scale, filled out by the participant).
  • Serious adverse events, that is, any untoward medical occurrence that results in death, is life threatening, requires hospitalisation or prolongation of existing hospitalisation, results in persistent or significant disability or incapacity, or is a congenital anomaly or birth defect (The International Conference on Harmonization (ICH) Guidelines for Good Clinical Practice (ICH_GCP 1997)).

Secondary outcomes

  • Mortality due to hepatitis B-related liver disease.
  • Non-serious adverse events. Any untoward medical occurrence in a participant or clinical investigation participant that does not meet the above criteria for a serious adverse event is defined as a non-serious adverse event.
  • Number of participants without histological improvement.
  • Number of participants with detectable HBsAg in serum or plasma.
  • Number of participants with detectable HBV DNA in serum or plasma. 

Exploratory outcomes

  • Number of participants with detectable HBeAg in serum or plasma (this outcome is only relevant for HBeAg-positive participants). 
  • Number of participants without HBeAg seroconversion in serum or plasma (this outcome is only relevant for HBeAg-positive participants). 
  • Number of participants without normalisation of transaminases.

References:
International Conference on Harmonisation Expert Working Group. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline. Guideline for Good Clinical Practice CFR & ICH Guidelines. Vol. 1. Philadelphia (PA): Barnett International/PAREXEL, 1997.

Core outcomes for chronic hepatitis C virus infection

Primary outcomes

  • All-cause mortality or hepatitis C-related morbidity (number of participants who developed cirrhosis, ascites, variceal bleeding, hepato-renal syndrome, hepatocellular carcinoma, or hepatic encephalopathy and who have not died). These outcomes will be tested as a composite outcome as well as individually (mortality or morbidity). Such composite outcomes need to be interpreted with caution, especially if the components are influenced differently by the intervention.
  • Health-related quality of life (any valid assessment scale, filled out by the participant).  
  • Serious adverse events, that is, any untoward medical occurrence that results in death, is life threatening, requires hospitalisation or prolongation of existing hospitalisation, results in persistent or significant disability or incapacity, or is a congenital anomaly or birth defect (The International Conference on Harmonization (ICH) Guidelines for Good Clinical Practice (ICH-GCP 1997)).

Secondary outcomes

  • Mortality due to hepatitis C-related liver disease.
  • Non-serious adverse events. Any untoward medical occurrence in a participant or clinical investigation participant that does not meet the above criteria for a serious adverse event is defined as a non-serious adverse events.
  • Number of participants without histological improvement.
  • Failure of virological response: number of participants without sustained virological response, i.e., number of participants with detectable hepatitis C virus RNA (i.e., above lower limit of detection) in the serum by a sensitive PCR-based essay or by a transcription-mediated amplification testing 12 and 24 weeks after end of treatment.

Exploratory outcomes

  • Number of participants without normalisation of transaminases.

References:
International Conference on Harmonisation Expert Working Group. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline. Guideline for Good Clinical Practice CFR & ICH Guidelines. Vol. 1. Philadelphia (PA): Barnett International/PAREXEL, 1997.

Information in general on core outcome sets can be found on www.comet-initiative.org/

If review authors wish to see more standard guidanace, please contact Dimitrinka Nikolova, the Managing Editor.