Resources for conducting evidence synthesis

College students working together at a computer

Resources for conducting evidence synthesis

Types of evidence synthesis review

This systematic Review Decision Tree figure was reproduced from Cornell University Library and the original is available here

What are the trade-offs for different types of evidence synthesis reviews?

Different types of evidence syntheses require different staffing resources and time and each type of review accomplishes a different purpose. Because of this, it is worthwhile to consider the goals you have for your review and the tradeoffs you might consider. This visual shows how evidence review products have different foci in terms of the scope of the review topic from narrow to broad, and the depth of the critical appraisal of the literature regarding quality and certainty of the evidence and the effectiveness of a given program or practice. Note that the purpose/question of a review and critical appraisal levels listed on the right hand side of the visual build on lower levels, for example, determining effectiveness also involves characterizing the extent of evidence.  

Another tool available is Right Review, which was designed to provide guidance and support for researchers on methods for the conduct and reporting of research syntheses. Right Review is a click-through interface that asks questions to help you select the best quantitative or qualitative research synthesis type.

] What are the trade-offs for different types of evidence synthesis reviews?

Image credit: Adapted Image was developed by Nicholas Parr, PhD, an affiliate investigator at the HEDCO Institute. 

Steps of evidence synthesis and associated resources 

Here are the steps we follow when conducting evidence syntheses at the HEDCO institute. These are largely drawn from Cochrane’s Handbook for Systematic Reviews of Interventions, which is open source and freely available.

1. Develop a Research Question

There are several existing question formulation frameworks you can use to develop the focal research question of your research synthesis. You can read more about these in this article by Andrew Booth and colleagues

  • PICO (Population, Intervention, Comparison, Outcome)
  • PICOTSS (Time, Setting, Study Design) 
  • SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type)
  • PerSPEcTiF (Perspective, Setting, Phenomenon of interest/problem, Environment, comparison, Time/timing, Findings)
2. Develop and Register a Protocol

An evidence synthesis protocol details your research question(s) and hypotheses, and your predetermined methods. Developing and publishing or registering your protocol prior to beginning your evidence synthesis improves the ability of others to replicate what you did in your evidence synthesis, reduces potential bias in the process, and assist evidence synthesis users to better understand the methods you used to conduct your evidence synthesis. It also improves confidence in your evidence synthesis findings. 

Protocol development guidance 

Is there any guidance on developing an evidence synthesis protocol?

Cochrane has developed guidance for each type of evidence synthesis and this includes components that should be specified in a preregistered protocol. 

Registering your protocol

Where can I register my protocol once it is developed?

  • You can register your protocol freely on Open Science Framework.
    • For an example of a preregistered protocol from the HEDCO Institute, you can click here
  • You can also register your protocol in the widely used PROSPERO registry, which is an International prospective register of systematic reviews.
3. Select Databases and Sources for Published and Unpublished Studies

The Campbell Guide: Searching for Studies (a) identifies the key issues faced by reviewers when gathering information for a review, (b) proposes different approaches in order to guide the work of the reviewer during the information retrieval phase, and (c) provides examples that demonstrate these approaches.

4. Write and Pilot a Search Strategy

The Peer Review of Electronic Search Strategies (PRESS) is a guide to developing replicable and comprehensive search strategies.

5. Article Screening


Guidance on best practices for abstract screening


A review on the advantages and drawbacks of different tools for screening abstracts in systematic reviews.

6. Risk of Bias Assessment

A Risk of Bias (RoB) assessment is an important step in any systematic review, meta-analysis, or overview of reviews (umbrella reviews). Because the individual studies that are part of a given evidence synthesis may contain varying degrees of bias, it is important to systematically assess and present that information so that evidence synthesis users can determine the limitations of a body of literature underlying a program, or practice. Further, there are steps you can take when conducting an evidence synthesis to reduce bias in the synthesis product. Thus, RoB assessments also exist to determine the extent of bias in evidence syntheses. Below are some tools you can use to systematically rate RoB. 

Risk of Bias for Individual Studies

  • RoB for Randomized Clinical Trials
  • Risk Of Bias In Non-Randomized Studies – of Interventions –

Risk of Bias for Evidence Syntheses

  • Risk of Bias in Systematic Reviews – ROBIS
  • Study Quality of Systematic Reviews – AMSTAR - 2

7. Quality/Certainty Appraisal

Grading of Recommendations Assessment, Development, and Evaluation (GRADE)

Distinct from a Risk of Bias assessment is the systematic rating of certainty in findings or in a body of evidence that has been evaluated through an evidence synthesis. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE)  approach can be used to rate the certainty and quality of evidence for each key finding from an evidence synthesis. Read more about GRADE here and here.


8. Data Extraction

9. Synthesize Results

These resources represent some of our team’s most used resources for synthesizing meta-analytic results. They are particularly useful for research syntheses in prevention science and other social science disciplines where programs and practices are often complex. 



10. Report Results

PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-analyses, and it's a set of guidelines that were designed to increase the quality of reporting and methodology for meta-analyses and systematic reviews. There are also several extensions for other types of reviews or meta-analyses. 

PRISMA Extensions 


Training on Research Synthesis Methods