Meta-Analysis

Meta-analysis is a research method that systematically combines information from a number of research studies to draw one or more conclusions from the body of available evidence. Meta-analysis is most useful in addressing questions for which a substantial body of high-quality research already exists, because the quality of the conclusions drawn by a meta-analysis depends in large part on the quality of the studies included. Meta-analysis can be particularly useful when studies have produced inconsistent or conflicting results. As the number of published research studies on a particular topic increases, meta-analysis is an important tool to help make sense of the available evidence.

Meta-analysis is sometimes referred to as an “analysis of analyses” because it draw conclusions by analyzing the results from a body of previous analyses. The concept of using mathematical methods to systematically examine the results of an existing body of studies, and to draw conclusions from those studies, dates back at least to the work of the British statistician Ronald A. Fisher in the first half of the twentieth century. However, meta-analysis did not become a common research technique until the late 1970s, following the 1977 publication by Mary Lee Smith and Gene V. Glass of a meta-analysis of hundreds of studies examining the outcomes of counseling and psychotherapy. Meta-analysis begins with a statement of the research question. For instance, does psychological counseling or psychotherapy have beneficial effects for those who receive it? The next step is to conduct research to find studies addressing the question. Ideally, a meta-analysis will include both published and unpublished studies in order to reduce the effects of publication bias, which refers to the tendency for studies that produced significant results to be published more frequently than studies that did not produce significant results.

Once a body of studies has been selected, they must be evaluated for quality, following a systematic set of procedures. For instance, is a particular research design or sample size required for a study to be included in the meta-analysis? This step is crucial, because the results of the meta-analysis can be quite different depending on the inclusion criteria applied. For this reason, a published meta-analysis generally includes a table describing all the studies located during the initial search, along with the inclusion criteria, the number of studies excluded, and the reasons for their exclusion. Sometimes two or more meta-analyses are performed addressing the same research question, using differing inclusion criteria in order to see how much the conclusions are affected by selection criteria. Finally, the results of the studies are synthesized quantitatively. Often, this is done by expressing the results of each study in terms of an effect size, a single number that includes the outcome of the study, the sample size, and the sample variability. The effect sizes from all the studies can then be analyzed, for instance to see how many found a positive effect and the average size of that effect.

—Sarah E. Boslaugh, MPH, PhD