• glossary
• meta-analysis

Bias, language: the systematic error introduced when the search of potential studies to be included in a meta-analysis is focused, in general, in one language (mainly English). Given that studies published in English are more widely read, it is possible that significant results of studies carried out in non-English countries are more published in English journals than in journals written in other languages.1

Bias, search: it is related to the language bias. It is the systematic error introduced when the search of studies is centred in just one database (for example, Medline). Journals written in English are overrepresented in Medline; furthermore, the journals from the country (and from neighbouring countries or either countries of similar language or culture) where the database is done are also represented in excess. It is recommended to consult more than one database (for example, Medline and Embase), supplemented by a hand search of the references of each publication collected.

Bias, publication: bias produced when the published studies do not represent adequately all the studies carried out on a specific topic. Many facts can origin this bias, although the best known is the trend to publish statistically significant (p < 0.05) or clinically relevant (high magnitude albeit non-significant) results. Other variables influencing publication bias are sample size (more in small studies), type of design (less in well conducted randomised controlled trials), funding, conflict of interest, prejudice against an observed association (for example, cocaine consumption and non-adverse effect on fetus), sponsorship.2 3

Cochrane Collaboration: an international organisation, named after Archibald Cochrane, which aims to help people make well informed decisions about healthcare by preparing, disseminating, and continuously updating systematic reviews of controlled clinical trials on the effects of healthcare interventions. These reviews are edited in the Cochrane Database of Systematic Reviews, available on CD-ROM.4 (More details on it can be found on the internet address www.cochrane.org).

Combining: see pooling.

Cumulative meta-analysis: A type of meta-analysis in which studies are sequentially pooled by adding each time one new study according to an ordered variable. For instance, if the ordered variable is the year of publication, studies will be ordered by it; then, a pooling analysis will be done every time a new article appears. It shows the evolution of the pooled estimate according to the ordered variable. Other common variables used in cumulative meta-analysis are the study quality, the risk of the outcome in the control group, the size of the difference between the groups, and other covariates (for example, mean time to treatment).5

DerSimonian-Laird's method: it was first described by Cochran in 1954. It was the first random effects model and applied to an additive model, based on the risk difference and weighted by the inverse of its variance.6

Effect size: a standardised scale free estimate of the relation between an exposure and an outcome. In a general sense, this term is applied to any measurement of the difference in the outcome between the study groups (so relative risk, odds ratio, and risk difference can be defined as “effect sizes”). When the effect size is applied to measurements of continuous variables (such as mean), its most common estimator is the standardised mean difference, which is calculated as the difference of means divided by the variability of the measures (the standard deviation). This is mainly useful when there is no common measure to all the studies.7

Egger's method: a procedure to detect publication bias. It consists in a simple linear regression of the effect size in a study divided by its standard error on the inverse of standard error and testing whether the intercept is statistically significant (at p < 0.1).8

File drawer problem: term coined by Rosenthal to mean the number of statistically non-significant studies (p > 0.05) that remain unpublished.9

Fixed effects model: any statistical model assuming homogeneity of effects across the studies being combined—that is, the true effect size has a common true value for all studies. In the summary estimate the variance of each studyi is taken into account only.

Funnel plot: a graphical method to display possible publication bias. It shows the relation between the effect size of study i and the size of the same study, which can be measured in different ways (standard error of the effect size, its inverse, sample size, or the number of effects observed in a study). If there is no publication bias, a typical symmetric funnel shape can be observed.

Glass, Gene V: The educational psychologist who coined the term “meta-analysis” in 1976.10

Heterogeneity: it means that there is between study variation. Many sources of heterogeneity can occur (more within observational epidemiology than within experimental designs): characteristics of the study population (for example, the underlying risk of the effect, different subgroups at high/low risk), variations in the study design (type of design, selection prodedures, sources of information, how the information is collected), different statistical methods, and different covariates adjusted for (if relevant). If heterogeneity exists the pooled effect size makes no sense, as it means that there may be more than one true effect sizes in the studies being combined. The main procedures to deal with heterogeneity are subgroup analysis, meta-regression, stratification, sensitivity analysis, cumulative meta-analysis, and identification of outliers.11

Homogeneity: the opposite to heterogeneity.

Inverse of variance: the most common kind of weight used to combine individual studies in a summary estimate. It can be applied to many effect sizes (relative risk, odds ratio, risk difference, proportions—like sensitivity or specificity—, etc) and in observational studies when adjusted effect sizes (odds ratio, relative risk) are provided.12 13

Mantel-Haenszel's method: a statistical method of pooling individual effect sizes (relative risk and odds ratio), first described to combine strata in an individual study. It is necessary to know the raw distribution of data, so it is in general appropriate for randomised controlled trials, where confounding is controlled for.14

Meta-analysis: the prefix “meta” means behind or beyond, of a higher or second order kind.15 It can be defined as a systematic identification, appraisal, synthesis, and, if relevant, statistical aggregation of all relevant prior studies on a specified topic according to a predetermined and explicit method.16

Meta-regression: a collection of statistical procedures (weighted/unweighted linear, logistic regression) to assess heterogeneity, in which the effect size of studyi is regressed on one or several covariates, with a value defined for each studyI.17

Overview: see meta-analysis.

Peto's method: a statistical method to combine individual studies, derived from the Mantel-Haenszel's procedure, in which the observed outcomes in the index group are compared with the expected ones, and weighting by its variance.18 It should only be used when the sample size of the arms of a study are balanced and the effect size is close to the null value.19

Pooling: the estimation of a summary (or pooled) effect size after the statistical aggregation of the individual studies.

Q statistic: a χ² test to assess heterogeneity across the studies included in a meta-analysis, in which the effect size of study i is compared with the pooled estimate. From the point of view of validity, power, and computational ease, this test of heterogeneity is the best choice.20

Qualitative meta-analysis: the part of meta-analysis concerning with the appraisal of the methods used in each individual study.

Quality of a study: the global appraisal of a study according to a validated and pretested protocol. Nevertheless, given that different scales yield divergent results, analyses based on a summary score of quality should be interpreted with caution.21 It is easier in clinical trials than in observational studies. Reviewers should be blind for the information that could influence the evaluation (authors, institutions, journal, direction of the association, etc).

Random effects model: a method of combining individual effect sizes in which heterogeneity is incorporated into the pooled estimate by including a between study component of variance. It assumes that the sample of studies included in the analysis is drawn from a population of studies. The random effects model does not assume homogeneity of the effects across the studies being pooled—that is, each sample of studies has a true effect size. There is no agreement on whether this model is more suitable than the fixed effects model to combine individual studies.

Research synthesis: see meta-analysis.

Sensitivity analysis: there are different ways of sensitivity analysis. In the assessment of heterogeneity it is concerned with the effects of inclusion and exclusion of specific studies. In the use of statistical procedures, sensitivity analysis is the repetition of the analysis using different statistical methods of pooling to assess whether the same results are achieved, and whether the quality of the individual studies and publication bias change the pooled estimates.

Systematic review: a synthesis of the results of several primary studies by using procedures that limit bias and random error. These procedures include a search of all potentially relevant investigations and the use of explicit and reliable criteria in the selection of investigations for review. A qualitative systematic review summarises the primary investigations without statistical pooling. Quantitative systematic review is synonymous with meta-analysis.22

Weight: the influence given to each individual study for the pooled analysis.