Background In 1967, Frederick Lord posed a conundrum that has confused scientists for over half a century. Since termed Lord’s ‘paradox’, the puzzle concerns the setting of analyses of change in an outcome measured at two times. In most studies, such data are examined either by analyzing the follow-up adjusted for baseline (Method 1) or analyzing the outcome ‘change score’ (Method 2). Alas, in observational data, these two approaches can return paradoxically different – even sign-discordant – results. Which, if either, is correct?
Methods At the heart of Lord’s ‘paradox’ lies another more general puzzle concerning the analyses of ‘change scores’ in observational data. By exploring the philosophy of change, and using directed acyclic graphs and data simulations, we examine the performance of the two standard approaches to analyses of change between two time points and discuss the wider lessons for contemporary data science.
Results The solution to Lord’s ‘paradox’ begins with recognising and distinguishing between the three reasons that a variable can change over time. First, endogenous (or ‘pre-determined’) change represents scale changes due to the realisation of past events. Second, random change represents all changes due to random or enigmatic processes. Third, exogenous change represents all non-random changes beyond what was expected.
In almost all situations, particularly where there is a wish to understand and/or influence change, exogenous change is the only element of interest. Alas, neither Method 1 nor Method 2 are suitable for robustly estimating the causes of exogenous change in observational data. For Method 1, this is because of susceptibility to mediator-outcome confounding and, with only two datapoints, there is insufficient data to distinguish exogenous change from random change. For Method 2, this is because analyses of ‘change scores’ evaluate obscure estimands with little, if any, real-world interpretation. Accurate and precise estimates instead require the use of appropriate causal inference methods (such as g-methods) and more than two measures of the outcome variable.
Conclusion Despite some eccentricities with his original example, Lord’s ‘paradox’ has surprising relevance to several aspects of contemporary data science. Like the Birthweight ‘paradox’ before it, the puzzle particularly demonstrates the benefits of considering causal questions within a formal causal framework and clearly identifying your estimand before conducting your data analysis. It also repeats the dangers of conducting and interpreting analyses of ‘change scores’ in observational data.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.