Systematic reviews of meta-analyses: applications and limitations

Systematic review of meta-analyses can be defined as the study working with meta-analysis as the unit of analysis. During the past years meta-analysis has been a growing field, more and more papers appearing on it with time. For instance, the number of references in a PubMed search using the keyword meta-analysis has steadily increased: 329 in 1990, 769 in 1995, 1649 in 2000, and 2545 in 2004. With such amount of papers it is not strange that the number of papers using meta-analysis itself as unit of analysis has also increased. From time to time the term “meta-meta-analysis” has appeared in scientific literature. The first time I read it in epidemiological literature was in an excellent editorial by Spitzer1 criticising the limitations of pooling data from observational studies. To follow properly the terminology currently applied the term meta-meta-analysis should be reserved for the combination of several meta-analyses in just one estimate. Katerndahl and Lawler2 used this procedure to analyse the variability of 23 meta-analyses of the benefits of cholesterol reduction in coronary heart disease. They concluded that methodologically better meta-analyses showed a trend to report more beneficial odds ratios and the main fact supporting cholesterol reduction was the statistical significance of the odds ratios for cardiovascular mortality. Grissom3 applied it to study in psychology the superiority of experimental treatments compared with placebo and control treatments, suggesting that there is more to therapeutic success than placebo effects and that placebo is typically better than do-nothing control …