‘First, do no harm’: are disability assessments associated with adverse trends in mental health? A longitudinal ecological study

Background In England between 2010 and 2013, just over one million recipients of the main out-of-work disability benefit had their eligibility reassessed using a new functional checklist—the Work Capability Assessment. Doctors and disability rights organisations have raised concerns that this has had an adverse effect on the mental health of claimants, but there are no population level studies exploring the health effects of this or similar policies. Method We used multivariable regression to investigate whether variation in the trend in reassessments in each of 149 local authorities in England was associated with differences in local trends in suicides, self-reported mental health problems and antidepressant prescribing rates, while adjusting for baseline conditions and trends in other factors known to influence mental ill-health. Results Each additional 10 000 people reassessed in each area was associated with an additional 6 suicides (95% CI 2 to 9), 2700 cases of reported mental health problems (95% CI 548 to 4840), and the prescribing of an additional 7020 antidepressant items (95% CI 3930 to 10100). The reassessment process was associated with the greatest increases in these adverse mental health outcomes in the most deprived areas of the country, widening health inequalities. Conclusions The programme of reassessing people on disability benefits using the Work Capability Assessment was independently associated with an increase in suicides, self-reported mental health problems and antidepressant prescribing. This policy may have had serious adverse consequences for mental health in England, which could outweigh any benefits that arise from moving people off disability benefits.


Discontinuities in the LFS health module.
Between quarter four 2009 and quarter one 2010 the ONS noted that there appeared to be a discontinuity in disability rates calculated from the QLFS. This was not due to any change in the questions and appears to have been due to the addition of a short introduction at the start of the health module: "I should now like to ask you a few questions about your health. These questions will help us estimate the number of people in the country who have health problems." This resulted in a small increase in the proportion of the population reporting health problems, but there was no change in the characteristics of this population and the ONS concluded that this increase was random.
[1] It therefore should not bias results in this analysis.
In 2013 Q2 the filter question identifying people with long term health problems was changed from: Do you have any health problems or disabilities that you expect will last for more than a year? [LNGLIM] To Do you have any physical or mental health conditions or illnesses lasting or expecting to last 12 months or more? [LNGLST] The questions referring to the types of health problems [HEAL0…HEAL17] remained the same.
To adjust for these changes in the questionnaire we included we included a dummy variable indicating the periods 2010q1 to 2013q1 and 2013q2-2013q4 in the regression model.

Correlation between self reported mental health problems and antidepressant prescribing rates.
We show below that local authority prevalence rates of mental health problems defined in this way correlates reasonably closely with antidepressant prescribing rates, both in terms of level and in terms of change over time. Figure 1 shows the prevalence in mental health problems for each local authority reported in the labour force survey correlated with the rate of antidepressant prescribing in each area.
We also find that the change in the prevalence of mental ill-health reported in an LA is associated with the change in the antidepressant prescribing rate (see Figure 2). As the estimates of the prevalence of mental ill-health are based on quite small samples in each LA, there is some degree of random measurement error, this is exacerbated in the differenced analysis in Figure 2. However even though there is quite a lot of random noise in the data the fact that we still find a relatively high level of correlation in Figure 2 indicates that these two indicators are measuring similar phenomena, namely the burden of diagnosed common mental health problems in the population. Specifically we estimated the following model: Eq 1: MHOUTCOME i,t = β 1 REASSESS ,I,t + β 2 UNEMP ,I,t + β 3 MEDWAGE ,I,t + β 4 GVA ,I,t + TIME1+TIME2+ β 5 IMDQ I, x TIME1 + β 6 GOR I x TIME1 +β 7 IMDQ I, x TIME2 + β 7 GOR I x TIME2 Where MHOUTCOME i,t is the mental health outcome in local authority i in time t as a rate per 100,000 population. REASSESS ,I,t is the cumulative percentage of the population who have experienced a reassessment in local authority i by time t. As the outcome is per 100,000 population this variable is reduced by a factor of 10, so that the coefficient reflects the number of additional cases of the mental health outcome per additional 10,000 people reassessed. UNEMP is the unemployment rate measured as the proportion of the working age population claiming unemployment benefits in local authority i in time t. LAEXPRATE is the total expenditure of local authority i in year t per head of population in £1000s. MEDWAGE is the median weekly full time gross wages in £100's in local authority i in time t. GVA if the Gross Value Added in £1000's for the region including local authority i in time t. IMDQ i is the quintile of deprivation of local authority i. GOR I is the government office region including local authority i. μ is a set of local authority dummy variables TIME1 is a time-trend term. (annual for suicide model and quarterly for self reported mental health problems and antidepressant models) TIME2 is an additional trend term (spline) to capture any change in trend from 2007. CONS is a constant. ε i,t is an error term

Multilevel logistic regression model.
To check whether the association of the reassessment rate with increases in self reported mental il-health, was influenced by changes in the composition of the population we estimated a multilevel model with the reassessment rate at the local authority level along with the quarterly unemployment rate, annual GVA, annual median wages, annual local authority expenditure, as well as a number of individual level control variables including age and sex, labour market status (employed, unemployed and inactive), number of physical chronic illnesses and socioeconomic group using the National Statistics Socio-economic Classification (NSSEC) groups. The model also included, interactions between sex and age, sex and labour market status and sex and number of physical comorbidities as these had differential effects by gender group.

Alternative adjustments for time trends.
In our main model we included data from 2004 in order to take into account trends in our outcomes prior to the implementation of the reassessment process. This is because preexisting trends could act as confounders, for example if trends in suicides were already increasing at a greater rate in areas of the country where the reassessment process proceeded more rapidly this may appear to be the result of the reassessment process if data prior to 2010 was not included. We allow time trends to varying before and after the economic crisis. This is because we know that declining trends in some mental health outcomes such as suicides reversed with the onset of the financial crisis. As there are potentially unobserved confounding factors that had differential trends across regions of the country before and after the recession we allowed underlying trends in mental health outcomes to vary by region and level of area deprivation. In a sensitivity analysis we estimate 3 additional models with simpler time trend structures finding that these tended to result in larger effect sizes, indicating that our preferred model is more conservative and potentially accounts for some unobserved confounders that follow similar time trends.
Model 1. Underlying time trends are assumed not to vary before and after the economic crisisi.e this model does not include a marginal spline for the 2007-2013 period. i.e MHOUTCOME i,t = β 1 REASSESS ,I,t + β 2 UNEMP ,I,t + β 3 MEDWAGE ,I,t + β 4 GVA ,I,t + TIME+ β 5 IMDQ I, x TIME + β 6 GOR I x TIME +β 7 IMDQ I, + CONS+ μ i + ε i,t Where TIME is a linear trend term, other variable names are as in Appendix 3.
Model 2. Underlying time trends are assumed not to vary before and after the economic crisis AND not to vary across levels of deprivation or regions. i.e MHOUTCOME i,t = β 1 REASSESS ,I,t + β 2 UNEMP ,I,t + β 3 MEDWAGE ,I,t + β 4 GVA ,I,t + TIME , + Model 3. The final model was the same as model 2, but was limited to data from 2010 onwards.
The association between the reassessment rate and each of the mental health outcomes estimated from each of these models are given below.

Models with alternative groups and outcomes.
To investigate if the association identified in our study was specific to mental health problems in the working age population we repeated the analysis using outcomes we would not expect to be influenced by the reassessment policy. Shadish et al. [2] refer to this as using Nonequivalent Dependent Variables (NDV) i.e those outcomes that should not be influenced by a change in the exposure but that could be influenced along with the outcome by unobserved confounding factors. Finding no effect on these outcomes can enhance the validity of observational analysis.
[2] We identified four Nonequivalent Dependent Variables in each of our datasets. Using the Quarterly Labour Force Survey we use the quarterly prevalence of mental health problems in the population over 65 years old and the prevalence of reported Heart, blood pressure & circulation problems in the working age population. Heart, blood pressure & circulation problems were selected as an NDV because it is unlikely that the reassessment process would increase the prevalence of these and this is the largest category of health problems reported in the QLFS. Therefore repeating our analysis with this outcome provides the greatest power to detect any associations. Heart, blood pressure & circulation problems are likely to be affected by other factors that could act as confounders or artifacts in our analysis, such as changes to survey design, changes in the propensity of people to report health problems, changes in access to healthcare, trends in physical health or other confounding factors that are associated with the reassessment rate and trends in this health outcomes. Similarly we investigated whether there was any association between the reassessment rate and trends in the rate of prescribing for cardiovascular conditions (BNF chapter 2). Finally we used data on suicides in over 65 year olds per 100,000 populations as an NDV. This outcome would be sensitive to any changes in the way that suicides are recorded as well as confounding factors that affect suicide risk across all age groups, which could have influenced our results. We find that the reassessment rate is not significantly associated with any of these Nonequivalent Dependent Variables (see Table 5) indicating that it is unlikely that the association that we find between the reassessment rate and trends in adverse mental health outcomes was due to confounding factors or artifacts that would also affect these Nonequivalent Dependent Variables.

Appendix web 5. Investigating variation in reassessment trends.
To make causal inferences about the association between the reassessment rate and trends in adverse mental health outcomes, we need to assume that the variation in local trends in the reassessment rate conditional on other covariates in our model was not associated with other causes of trends in mental health outcomes during this time. In other words we assume that the variation is as good as random. There are a number of reasons that might account for variation in trends in the reassessment rate across local areas. Firstly there is the targeting of the programme at more deprived areas and regions with higher levels of people on Incapacity Benefits, secondly there are logistical, human resource and planning considerations that affect variation in implementation of any large-scale operation. The first of these we control for by including fixed (local authority) effects in the model and separate times trends by area deprivation and region. The remaining variation is therefore likely to be due to these logistical, human resource and planning considerations. We know that there was considerable variation in the implementation process, with some assessment centres progressing at a slower rate than othersleading to a large backlog of claims at some centres. Reports of the reasons for this variation include, technical problems, under estimates of referral rates and the time involved in carrying out assessments when planning resources and problems with recruiting staff [3][4][5][6][7].
To further investigate this variation in reassessment rates we estimate a fixed effects regression model with reassessment trends as the outcome, including the main variables used in the analysis. See table 1. We can see that the reassessment progressed at a faster rate in the North East and North West, in more deprived areas than in more affluent areas, the trend in reassessment was also negatively associated with trends in unemployment, wages and trends in local government expenditure. This indicates that it was necessary to control for these trends in our analysis to reduce possible sources of bias. We further investigated whether trends in the reassessment rate were additionally associated with trends in initial reassessment rates for Employment Support Allowance in each area and whether the level of rurality in a local authority area influenced the trend in reassessments. It is possible that as the same organisation (ATOS) was carrying out initial assessments during this time high demand of initial assessments in an area may have reduced the rate at which the reassessment programme progressed, it is also possible that logistical constraints on the programme were greater in more rural areas with more dispersed populations. We divided the local authorities into 5 groups based on the proportion of the population in each LA that was living in a rural area according to Office for National Statistics rural/urban classifications and added interaction terms between level of rurality and time into the model. Regional quarterly caseloads of initial assessments for ESA as a percentage of the working age population were used to assess trends in initial assessment rates. Adding these terms to the model indicated that there was no significant difference in trends in reassessment between more rural or more urban areas, when other covariates were taken into account. However the trend in reassessments was significantly negatively associated with the trend in initial reassessment in an area i.e the reassessment process tended to proceed at a slower rate in areas were there was a greater increase in initial assessments. To investigate the geographical pattern of the variation in the reassessment rate that was not explained by our control variables we have mapped the average residuals for each local authority area from the model above (see Figure below). This indicates the variation in the reassessment rate after accounting for the control variables in our model. There is no obvious spatial pattern to this variation, supporting the assumption that it is approximately random. We finally assessed whether including regional trends in initial ESA assessments and separate trends by level of rurality in our model for mental health outcomes affected our results. Local trends in initial assessments for ESA and separate trends by level of rurality were not significantly associated with local trends in any of our mental health outcomes and adding the term to our main models did not change the association between the reassessment rate and the mental health outcomes. (see Table 6 and 7)  Appendix web 6. Predicted trends in mental health outcomes in the presence and absence of the reassessment policy by level of area deprivation.
We used out regression models to estimate how the predicted trends of our mental health outcomes would have differed in the absence of the reassessment policy compared to trends in the presence of the policy. To assess the potential impact on health inequalities, we investigated whether the association between the reassessment rate and the mental health outcomes varied by level of baseline deprivation by testing interactions between these variables, and estimated the trends in the most affluent and most deprived parts of the country based on the upper and lower quintiles of area deprivation (IMD). As the relationship between deprivation and antidepressant prescribing is very different within London as compared to areas outside London[1] we presented results for antidepressant prescribing separately for these areas. Figure 5 shows the estimated trends in each mental health outcome in the most deprived and least deprived areas of England and the predicted trend that would have been expected from the regression models if these 1.03 million people had not been through this reassessment process. There was no significant interaction between the reassessment rate and area deprivation, i.e the same level of increase in the reassessment rate was associated with the same impact in deprived areas as in more affluent areas. However as more disadvantaged socioeconomic groups are more likely to be in receipt of disability benefits, and thus to be assessed, the reassessment policy was associated with a greater increase in these adverse mental health outcomes in more deprived areas. Our analysis shows that the gap in the suicide rate and to a lesser extent self reported mental health problems between the least deprived and most deprived areas had been declining prior to the introduction of the reassessment policy, however after the policy this trend reverses. This suggests that there would have been a further narrowing of these inequalities in the absence of the reassessment process.