Article Text

Download PDFPDF
OP09 Lord’s ‘paradox’ explained: The 50-year warning on the analyses of ‘change scores’*
  1. Peter WG Tennant1,2,3,
  2. Georgia D Tomova1,2,3,
  3. Kellyn F Arnold1,4,
  4. Mark S Gilthorpe1,2,3
  1. 1Leeds Institute for Data, University of Leeds, Leeds, UK
  2. 2School of Medicine, University of Leeds, Leeds, UK
  3. 3Alan Turing Institute, London, UK
  4. 4School of Geography, University of Leeds, Leeds, UK

Abstract

Background In 1967, Frederick Lord posed a conundrum that has confused scientists for over half a century. Since termed Lord’s ‘paradox’, the puzzle concerns the setting of analyses of change in an outcome measured at two times. In most studies, such data are examined either by analyzing the follow-up adjusted for baseline (Method 1) or analyzing the outcome ‘change score’ (Method 2). Alas, in observational data, these two approaches can return paradoxically different – even sign-discordant – results. Which, if either, is correct?

Methods At the heart of Lord’s ‘paradox’ lies another more general puzzle concerning the analyses of ‘change scores’ in observational data. By exploring the philosophy of change, and using directed acyclic graphs and data simulations, we examine the performance of the two standard approaches to analyses of change between two time points and discuss the wider lessons for contemporary data science.

Results The solution to Lord’s ‘paradox’ begins with recognising and distinguishing between the three reasons that a variable can change over time. First, endogenous (or ‘pre-determined’) change represents scale changes due to the realisation of past events. Second, random change represents all changes due to random or enigmatic processes. Third, exogenous change represents all non-random changes beyond what was expected.

In almost all situations, particularly where there is a wish to understand and/or influence change, exogenous change is the only element of interest. Alas, neither Method 1 nor Method 2 are suitable for robustly estimating the causes of exogenous change in observational data. For Method 1, this is because of susceptibility to mediator-outcome confounding and, with only two datapoints, there is insufficient data to distinguish exogenous change from random change. For Method 2, this is because analyses of ‘change scores’ evaluate obscure estimands with little, if any, real-world interpretation. Accurate and precise estimates instead require the use of appropriate causal inference methods (such as g-methods) and more than two measures of the outcome variable.

Conclusion Despite some eccentricities with his original example, Lord’s ‘paradox’ has surprising relevance to several aspects of contemporary data science. Like the Birthweight ‘paradox’ before it, the puzzle particularly demonstrates the benefits of considering causal questions within a formal causal framework and clearly identifying your estimand before conducting your data analysis. It also repeats the dangers of conducting and interpreting analyses of ‘change scores’ in observational data.

  • Data science
  • Methodology
  • Longitudinal Studies

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.