Table 2.

Questions to ask in the evaluation of systematic reviews.1

• Can you find an important (clinical) question that the review addressed?
 –Must be defined precisely for the original reviewer to include or reject a paper
• Was a thorough search done of the appropriate databases, and were other potentially important sources explored?
 –Recognizing limitations of MEDLINE, were other databases explored
 –Was there access to references listed in primary sources
 –Unpublished sources, e.g., theses, internal reports, non-peer-reviewed journals, industry
 –Raw data from published trials
 –Foreign language literature
• Was methodologic quality assessed and were the trials weighted accurately?
 –Extent to which the design and the conduct prevented bias
 –Width of the confidence intervals (precision)
 –Generalizibility of the results, applicability to a target population (external validity)
• How sensitive are the results to the way the review was done?
 –Would the conclusions change if the inclusion criteria had been changed, unpublished studies had been excluded, or all of the patients unaccounted for were assumed to have died, or alternatively to have been cured (sensitivity analysis)
• Have the numeric results been interpreted with common sense and with due regard to the broader aspects of the problem?
 –No matter how significant a result, how will it influence the care of a single patient
 –The external validity or relevance of the trials that had been included