Information for authors

On  the  Cochrane training website page authors of systematic reviews will find everything they need to conduct their reviews https://training.cochrane.org/online-learning/author-guidelines

At a CHBG Editorial Team Group meeting in Copenhagen in April 2009, CHBG editors decided that CHBG review authors should be encouraged to use trial sequential analyses of their important meta-analyses. In the recent years, the central Cochrane editorial team suggested the use of Trial Sequential Analysis only as a sensitivity analysis of imprecision and compare this assessment with the assessment of imprecision with GRADE (Thomas 2019).

The Trial Sequential Analysis (TSA) software is free to download and use. The Trial Sequential Analysis Manual can be downloaded from the TSA website. Review authors should follow the instructions at www.ctu.dk/tools-and-links/tools-and-links.aspx when creating images for publication in their reviews with RevMan web.  

The following is an example of Trial Sequential Analysis text: 

"Sensitivity analysis

In addition to the sensitivity analysis described in the 'Dealing with missing data' section, we will perform sensitivity analysis on trials at low risk of bias (or both). We also plan to assess imprecision with Trial Sequential Analysis (see below), using also the eight-step procedure for validation of meta-analytic results in systematic reviews as suggested by Jakobsen and colleagues (Jakobsen 2014).

Trial Sequential Analysis

Cumulative meta-analyses are at risk of producing random errors due to sparse data and multiple testing of accumulating data (Brok 2008; Wetterslev 2008; Brok 2009; Thorlund 2009; Wetterslev 2009; Thorlund 2010; Wetterslev 2017); therefore, Trial Sequential Analysis (TSA 2011) can be applied as a secondary analysis to control this risk (Thorlund 2011; Thomas 2019). We will use Trial Sequential Analysis as a sensitivity analysis to our GRADE assessments of imprecision. The former is taking meta-analytic model and diversity into consideration whereas the latter is based on a fixed-effect model and ignores diversity. The required information size (i.e. the number of participants needed in a meta-analysis to detect or reject a certain intervention effect) can be calculated in order to control random errors (Wetterslev 2008; Wetterslev 2009; Wetterslev 2017). The required information size takes into account the event proportion in the control group, the assumption of a plausible relative risk reduction, and the heterogeneity of the meta-analysis (Wetterslev 2008; Wetterslev 2009; Turner 2013; Wetterslev 2017). Trial Sequential Analysis enables testing for significance to be conducted each time a new trial is included in the meta-analysis. On the basis of the required information size, trial sequential monitoring boundaries can be constructed. This enables one to determine the statistical inference concerning cumulative meta-analysis that has not yet reached the required information size (Wetterslev 2008).

If the trial sequential monitoring boundary is crossed by the cumulative Z-curve before reaching the required information size, we may conclude that sufficient evidence is collected to validly assess benefit or harm, and that inclusion of additional trial data may be redundant. In contrast, if the boundaries for benefit or harm are not crossed, we may conclude that further trials are necessary before a certain intervention effect can be evaluated. Trial Sequential Analysis also allows for assessment of the sufficiency of evidence for a postulated intervention effect. A lack of effect is evident if the cumulative Z-score crosses the trial sequential monitoring boundaries for futility.

We will make relatively conservative estimations of the anticipated intervention effect to control the risks of random error (Jakobsen 2014). Large, anticipated intervention effects lead to small required information sizes, and the thresholds for significance will be less strict after the information size has been reached (Jakobsen 2014).

We will analyse all primary and secondary outcomes using Trial Sequential Analysis. These analyses will allow us to calculate the Trial Sequential Analysis-adjusted CIs based on the following assumptions.

Primary outcomes

We will estimate the diversity-adjusted required information size (Wetterslev 2009), based on the proportion of participants with an outcome in the control group. We will use an alpha of 0.025 because of our three primary outcomes, a beta of 10%, and the diversity suggested by the trials in the meta-analysis (Jakobsen 2014; Castellini 2017).

As anticipated intervention effects for the primary outcomes in the Trial Sequential Analysis, we will use the following.

All-cause mortality: a relative risk reduction of 10% and the observed proportion of mortality in the control group.

Serious adverse events: a relative risk reduction of 20% and the observed proportion of serious adverse events in the control group.

Health-related quality of life: minimal relevant difference observed SD divided by two.

Secondary outcomes

We will estimate the diversity-adjusted required information size (Wetterslev 2009), based on the proportion of participants with an outcome in the control group when analysing dichotomous outcomes, and we will use the observed SD when analysing continuous outcomes. We will use an alpha of 0.033 because of the two secondary outcomes, a beta of 10%, and the diversity suggested by the trials in the meta-analysis (Jakobsen 2014; Castellini 2017).

As anticipated intervention effects for the secondary outcomes in the Trial Sequential Analysis, we will use the following relative risk reductions or increases.

Sepsis: a relative risk reduction of 20% and the observed incidence of failure treatment in the control group.

Non-serious adverse events: a relative risk reduction of 20%.

Assessment of imprecision

In order to have a better judgement of imprecision in the included trials, we will compare GRADE and Trial Sequential Analysis results regarding our Primary outcomes and Secondary outcomes (Castellini 2018; Gartlehner 2019; Thomas 2019).

Assessment of significance

We will assess the intervention effects using the random-effects model meta-analysis (DerSimonian 1986). For analysis of the three primary outcomes, we will consider significant a P value less than 0.025 (Jakobsen 2014), as this will secure a family-wise error rate (FWER) below 0.05. We will apply an eight-step procedure to assess if the results from the meta-analyses have passed the thresholds for significance (Jakobsen 2014).

Reporting of reviews

For policies on the reporting of reviews (for example on the discussion of results, the use of tables and figures, and the naming of studies), authors must follow the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions.

References suggested for use in systematic reviews follow below.

Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events. JAMA 2003;290:921-8.

Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. Journal of Clinical Epidemiology 2011;64(4):401-6.

Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics 1994;50:(4):1088-101.

Chan AW, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al. SPIRIT 2013 Statement: defining standard protocol items for clinical trials. Annals of Internal Medicine 2013;158:200-7.

Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin J, Dickersin K, Hróbjartsson A, Schulz KF, Parulekar WR, Krleža-Jerić K, Laupacis A, Moher D. SPIRIT 2013 Explanation and Elaboration: Guidance for protocols of clinical trials. BMJ 2013;346:e7586.

Deeks JJ, Higgins JPT, Altman DG (editors). Chapter 10: Analysing data and undertaking meta-analyses. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane, 2019. Available from www.training.cochrane.org/handbook.

DeMets DL. Methods of combining randomized clinical trials: strengths and limitations. Statistics in Medicine 1987;6(3):341-50.

DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials 1986;7(3):177-88.

Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technology Assessment 2003;7:1-76.

Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple graphical test. BMJ (Clinical Research Ed.) 1997;315(7109):629-34.

Fisher RA. On the interpretation of χ2 from contingency tables, and the calculation of P. Journal of the Royal Statistical Society 1922;85(1):87-94. 

Garattini S, Jakobsen JC, Wetterslev J, Bertelé V, Banzi R, Rath A, et al. Evidence-based clinical practice: overview of threats to the validity of evidence and how to minimise them. European Journal of Internal Medicine 2016;32:13-21. 

Gluud LL, Thorlund K, Gluud C, Woods L, Harris R, Sterne JA. Correction: reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine 2008;149(3):219. 

GRADEpro GDT: GRADEpro Guideline Development Tool [Software]. McMaster University, 2015 (developed by Evidence Prime, Inc.). Available from gradepro.org.

Schünemann H, Brożek J, Guyatt G, Oxman A, editors. GRADE handbook for grading quality of evidence and strength of recommendations. Updated October 2013. The GRADE Working Group, 2013. Available fromguidelinedevelopment.org/handbook. When referring to a specific chapter or subsection refer to it by the title and section number, not page numbers. Example: Chapter authors in Schünemann H, Brożek J, Guyatt G, Oxman A, editors. GRADE handbook for Grading quality of evidence and strength of recommendations. Version XX [updated XX 2014].Guyatt G, Andrews J, Oxman AD, Alderson P, Dahm P, Falck-Ytter Y, et al. GRADE guidelines: 15. Going from evidence to recommendations: the significance and presentation of recommendations. Journal of Clinical Epidemiology 2013;66(7):719-25.

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. Journal of Clinical Epidemiology 2011;64(4):383-94.

Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, et al. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. Journal of Clinical Epidemiology 2013;66(2):151-7.

Guyatt GH, Ebrahim S, Alonso-Coello P, Johnston BC, Mathioudakis AG, Briel M, et al. GRADE guidelines 17: assessing the risk of bias associated with missing participant outcome data in a body of evidence. Journal of Clinical Epidemiology 2017;87:14-22.

Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. Journal of Clinical Epidemiology 2011;64(4):395-400.

Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, et al. GRADE guidelines 6. Rating the quality of evidence--imprecision. Journal of Clinical Epidemiology 2011;64(12):1283-93.

Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 7. Rating the quality of evidence--inconsistency. Journal of Clinical Epidemiology 2011;64(12):1294-302.

Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidence--indirectness. Journal of Clinical Epidemiology 2011;64(12):1303-10.

Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence--publication bias. Journal of Clinical Epidemiology 2011;64(12):1277-82.

Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, et al. GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes. Journal of Clinical Epidemiology 2013;66(2):158-72.

Guyatt GH, Oxman AD, Sultan S, Glasziou P, Akl EA, Alonso-Coello P, et al. GRADE guidelines: 9. Rating up the quality of evidence. Journal of Clinical Epidemiology 2011;64(12):1311-6.

Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias). Journal of Clinical Epidemiology 2011;64(4):407-15.

Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, et al. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. Journal of Clinical Epidemiology 2013;66(2):173-83.

Core outcomes for chronic hepatitis B (CHB) virus infection

Primary outcomes

  • All-cause mortality or hepatitis B-related morbidity (number of participants who developed cirrhosis, ascites, variceal bleeding, hepato-renal syndrome, hepatocellular carcinoma, or hepatic encephalopathy and who have not died). These outcomes will be tested as a composite outcome as well as individually (mortality or morbidity). Such composite outcomes need to be interpreted with caution, especially if the components are influenced differently by the intervention.
  • Health-related quality of life (any valid assessment scale, filled out by the participant).
  • Serious adverse events, that is, any untoward medical occurrence that results in death, is life threatening, requires hospitalisation or prolongation of existing hospitalisation, results in persistent or significant disability or incapacity, or is a congenital anomaly or birth defect (The International Conference on Harmonization (ICH) Guidelines for Good Clinical Practice (ICH_GCP 1997)).

Secondary outcomes

  • Mortality due to hepatitis B-related liver disease.
  • Proportion of people with adverse events considered nonserious (any untoward medical occurrence in a participant or clinical investigation participant, that does not meet the above criteria for a serious adverse event, is defined as a non-serious adverse effect).
  • Proportion of people without histological improvement.
  • Proportion of people with detectable HBV-DNA in serum or plasma.
  • Proportion of people with detectable HBsAg in serum or plasma.
  • Proportion of people with detectable HBeAg in serum or plasma (this outcome is only relevant for HBeAg-positive participants). 
  • Proportion of people without HBeAg seroconversion in serum or plasma (this outcome is only relevant for HBeAg-positive participants). 
  • Proportion of people without normalisation of transaminases (i.e. biochemical response).

References:

International Conference on Harmonisation Expert Working Group. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline. Guideline for Good Clinical Practice CFR & ICH Guidelines. Vol. 1. Philadelphia (PA): Barnett International/PAREXEL, 1997.

Core outcomes for chronic hepatitis C virus infection

Primary outcomes

  • All-cause mortality or hepatitis C-related morbidity (number of participants who developed cirrhosis, ascites, variceal bleeding, hepato-renal syndrome, hepatocellular carcinoma, or hepatic encephalopathy and who have not died). These outcomes will be tested as a composite outcome as well as individually (mortality or morbidity). Such composite outcomes need to be interpreted with caution, especially if the components are influenced differently by the intervention.
  • Health-related quality of life (any valid assessment scale, filled out by the participant).  
  • Serious adverse events, that is, any untoward medical occurrence that results in death, is life threatening, requires hospitalisation or prolongation of existing hospitalisation, results in persistent or significant disability or incapacity, or is a congenital anomaly or birth defect (The International Conference on Harmonization (ICH) Guidelines for Good Clinical Practice (ICH-GCP 1997)).

Secondary outcomes

  • Mortality due to hepatitis C-related liver disease.
  • Non-serious adverse events. Any untoward medical occurrence in a participant or clinical investigation participant that does not meet the above criteria for a serious adverse event is defined as a non-serious adverse events.
  • Number of participants without histological improvement.
  • Failure of virological response: number of participants without sustained virological response, i.e., number of participants with detectable hepatitis C virus RNA (i.e., above lower limit of detection) in the serum by a sensitive PCR-based essay or by a transcription-mediated amplification testing 12 and 24 weeks after end of treatment.

Exploratory outcomes

  • Number of participants without normalisation of transaminases.

References:

International Conference on Harmonisation Expert Working Group. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline. Guideline for Good Clinical Practice CFR & ICH Guidelines. Vol. 1. Philadelphia (PA): Barnett International/PAREXEL, 1997.

Information on core outcome sets can be found on www.comet-initiative.org/