![College students working together at a computer](/sites/default/files/styles/custom_xl/public/2024-05/resources-2-hero-1440x-400.jpg?itok=b3NZxyfH)
Resources for conducting evidence synthesis
Types of evidence synthesis review
This systematic Review Decision Tree figure was reproduced from Cornell University Library and the original is available here.
What are the trade-offs for different types of evidence synthesis reviews?
Different types of evidence syntheses require different staffing resources and time and each type of review accomplishes a different purpose. Because of this, it is worthwhile to consider the goals you have for your review and the tradeoffs you might consider. This visual shows how evidence review products have different foci in terms of the scope of the review topic from narrow to broad, and the depth of the critical appraisal of the literature regarding quality and certainty of the evidence and the effectiveness of a given program or practice. Note that the purpose/question of a review and critical appraisal levels listed on the right hand side of the visual build on lower levels, for example, determining effectiveness also involves characterizing the extent of evidence.
Another tool available is Right Review, which was designed to provide guidance and support for researchers on methods for the conduct and reporting of research syntheses. Right Review is a click-through interface that asks questions to help you select the best quantitative or qualitative research synthesis type.
![] What are the trade-offs for different types of evidence synthesis reviews?](/sites/default/files/styles/custom_xl/public/2024-05/scope-of-review.png?itok=FQNQlflm)
Image credit: Adapted Image was developed by Nicholas Parr, PhD, an affiliate investigator at the HEDCO Institute.
Steps of evidence synthesis and associated resources
Here are the steps we follow when conducting evidence syntheses at the HEDCO institute. These are largely drawn from Cochrane’s Handbook for Systematic Reviews of Interventions, which is open source and freely available.
There are several existing question formulation frameworks you can use to develop the focal research question of your research synthesis. You can read more about these in this article by Andrew Booth and colleagues.
- PICO (Population, Intervention, Comparison, Outcome)
- PICOTSS (Time, Setting, Study Design)
- SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type)
- PerSPEcTiF (Perspective, Setting, Phenomenon of interest/problem, Environment, comparison, Time/timing, Findings)
An evidence synthesis protocol details your research question(s) and hypotheses, and your predetermined methods. Developing and publishing or registering your protocol prior to beginning your evidence synthesis improves the ability of others to replicate what you did in your evidence synthesis, reduces potential bias in the process, and assist evidence synthesis users to better understand the methods you used to conduct your evidence synthesis. It also improves confidence in your evidence synthesis findings.
Protocol development guidance
Is there any guidance on developing an evidence synthesis protocol?
Cochrane has developed guidance for each type of evidence synthesis and this includes components that should be specified in a preregistered protocol.
Registering your protocol
Where can I register my protocol once it is developed?
- You can register your protocol freely on Open Science Framework.
- You can also register your protocol in the widely used PROSPERO registry, which is an International prospective register of systematic reviews.
The Campbell Guide: Searching for Studies (a) identifies the key issues faced by reviewers when gathering information for a review, (b) proposes different approaches in order to guide the work of the reviewer during the information retrieval phase, and (c) provides examples that demonstrate these approaches.
The Peer Review of Electronic Search Strategies (PRESS) is a guide to developing replicable and comprehensive search strategies.
Guidance
Guidance on best practices for abstract screening.
Software
A review on the advantages and drawbacks of different tools for screening abstracts in systematic reviews.
A Risk of Bias (RoB) assessment is an important step in any systematic review, meta-analysis, or overview of reviews (umbrella reviews). Because the individual studies that are part of a given evidence synthesis may contain varying degrees of bias, it is important to systematically assess and present that information so that evidence synthesis users can determine the limitations of a body of literature underlying a program, or practice. Further, there are steps you can take when conducting an evidence synthesis to reduce bias in the synthesis product. Thus, RoB assessments also exist to determine the extent of bias in evidence syntheses. Below are some tools you can use to systematically rate RoB.
Risk of Bias for Individual Studies
- RoB for Randomized Clinical Trials
- Individually Randomized Trials – RoB 2
- Cluster Randomized Trials – RoB 2 Cluster
- Risk Of Bias In Non-Randomized Studies – of Interventions –
Risk of Bias for Evidence Syntheses
- Risk of Bias in Systematic Reviews – ROBIS
- Study Quality of Systematic Reviews – AMSTAR - 2
7. Quality/Certainty Appraisal
Grading of Recommendations Assessment, Development, and Evaluation (GRADE)
Distinct from a Risk of Bias assessment is the systematic rating of certainty in findings or in a body of evidence that has been evaluated through an evidence synthesis. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach can be used to rate the certainty and quality of evidence for each key finding from an evidence synthesis. Read more about GRADE here and here.
8. Data Extraction
Guidance
These resources represent some of our team’s most used resources for synthesizing meta-analytic results. They are particularly useful for research syntheses in prevention science and other social science disciplines where programs and practices are often complex.
Methods
Software
PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-analyses, and it's a set of guidelines that were designed to increase the quality of reporting and methodology for meta-analyses and systematic reviews. There are also several extensions for other types of reviews or meta-analyses.
PRISMA Extensions
Training on Research Synthesis Methods
- Cochrane Interactive Learning
- Campbell Collaboration Training on systematic reviews and meta-analysis, which is currently being offered freely during the pilot phase.
- Campbell Collaboration Training Resources