Article Text
PDF
Abstract
Background Reductions in the eligibility requirements and generosity of disability benefits have been introduced in several Organisation for Economic Cooperation and Development (OECD) countries in recent years, on the assumption that this will increase work incentives for people with chronic illness and disabilities. This paper systematically reviews the evidence for this assumption in the context of well-developed welfare systems.
Method Systematic review of all empirical studies from five OECD countries from 1970 to December 2009 investigating the effect of changes in eligibility requirements or level of disability benefits on employment of disabled people.
Results Sixteen studies were identified. Only one of five studies found that relaxed eligibility was significantly associated with a decline in employment. The most robust study found no significant effect. On generosity, eight out of 11 studies reported that benefit levels had a significant negative association with employment. The most robust study demonstrated a small but significant negative association.
Conclusion There was no firm evidence that changes in benefit eligibility requirements affected employment. While there was some evidence indicating that benefit level was negatively associated with employment, there was insufficient evidence of a high enough quality to determine the extent of that effect. Policy makers and researchers need to address the lack of a robust empirical basis for assessing the employment impact of these welfare reforms as well as potentially wider poverty impacts.
- Disability insurance
- disability SI
- disabled persons
- employed CG
- employment
- social inequalities
- social welfare
- socioeconomic factors
Statistics from Altmetric.com
- Disability insurance
- disability SI
- disabled persons
- employed CG
- employment
- social inequalities
- social welfare
- socioeconomic factors
Dramatic rises in the number of people claiming disability benefits in several Organisation for Economic Cooperation and Development (OECD) countries over recent decades have led to concerns about the social and economic exclusion of disabled people and the costs of income support for these groups.1–5 Evidence from the UK and Sweden indicates a social gradient in the employment of chronically ill and disabled people, with employment rates declining with decreasing socioeconomic status.6 7 Worklessness increases the risk of poverty and social exclusion, which may further damage health and exacerbate health inequalities.
One aim of disability policy is to provide adequate income security to people with a health condition or impairment when they are too ill to work. Economists have long debated the effects of welfare programmes, with some arguing that the level of provision of income security benefits themselves acts as a disincentive to labour force participation.8 Several authors in the USA have concluded that the increase in the availability of disability benefits is responsible for most of the decline in labour force participation among older men in that country.9–11 These econometric studies have, however, been criticised for inaccurately estimating the disincentive effects of disability benefits.12–14
The empirical evidence that does exist to support the hypothesis that disability benefits are major disincentives for work largely comes from studies in the USA, but it would be unsound to generalise from the US context to countries with more extensive welfare systems. The consequences for disabled people of not being employed are very different in the USA, where there are fewer safety nets, no universal healthcare system, and health insurance is often provided through an employer and is lost when a person loses that employment. There is a need to synthesise the evidence on the question of disincentives in the context of advanced welfare systems, which is the aim of this paper.
We conducted a systematic review of the evidence from five countries (UK, Canada, Norway, Sweden and Denmark) with well-developed health and welfare systems to answer the following review question: ‘To what extent does variation in the generosity or eligibility requirements of disability benefit programmes affect labour market participation?’ These countries represent both liberal (UK, Canada) and social democratic (Norway, Sweden and Denmark) welfare-state regimes as outlined in the typology developed by Esping-Andersen.15 While these two types of welfare-state regime reflect very different overall disability benefit levels and coverage, these five countries share common ground in terms of the health policy context for working-age disabled people and their employment situation. All five countries, for instance, have universal healthcare systems and health insurances/eligibility coverage that is not linked to employment, so that sick and disabled people are protected from catastrophic healthcare costs when they fall ill, and do not lose health coverage if their ill-health means that they leave their job. These countries have all implemented numerous policies over the past 30 years focused on return to work for chronically ill and disabled people. These have included reforms to the benefit system to alter benefit generosity and eligibility (see appendix 1, available online only), providing an opportunity to exploit these natural policy experiments. More recently, policy makers in these countries have begun to experiment with reducing the generosity and narrowing the eligibility criteria for these benefits, on the assumption that this will increase the employment of people with chronic illness and disabilities. This strategy underpins the introduction of the employment support allowance in the UK in 2008, the 2008 reforms of the Swedish sickness insurance system, the 2003 reforms of disability benefits in Denmark and the 2004 disability benefit reforms in Norway (see appendix 1).3–5 16 While there has been a traditional review assessing the factors that have contributed to recent increases in disability benefits recipients in the UK,17 to our knowledge this is the first systematic review to address this issue and to take into account the relevance of the welfare system context.
Methods
Through our search and selection strategy we sought to identify all empirical studies from Canada, Denmark, Norway, Sweden and the UK that addressed the research question: ‘To what extent does variation in the generosity or eligibility requirements of disability benefit programmes affect labour market participation?’ We restricted our review to studies from these countries for comparability and pragmatic reasons. These countries meet three key criteria: universal healthcare systems; health insurance coverage/eligibility not linked to employment and well-developed social protection systems. This means that while a variety of policy approaches have been taken both in terms of the level and universality of disability benefits, the health and welfare context for working-age disabled people is sufficiently similar to allow relevant policy learning from one country to another. We acknowledge that there may be other countries that meet the above three criteria but for pragmatic reasons these were not included. This is because we believe that an in-depth understanding of the context of policy implementation is essential for the valid interpretation of the evidence. The authors are based in and have the required detailed knowledge of welfare policy in the five selected countries.
Searches
We searched 13 databases (appendix 2, available online only) from 1970 to December 2009. In addition, grey literature searches were conducted on 12 relevant governmental and non-governmental organisational web sites. A comprehensive list of linked search terms was used, with terms associated with the policy, the population and the outcome (appendix 2, available online only). Websites were searched using a search engine that allowed for site-specific searches with multiple search terms linked with Boolean commands.18
Selection
The searches identified 3730 potentially relevant studies. Following selection using the inclusion criteria in box 1 and validity assessment detailed in appendix 3 (available online only), a total of 16 studies was included in the final review (figure 1).
Box 1 Criteria used to select studies for data collection and validity assessment
Study design
All quantitative study designs.
Participants/population
Working age (16–69 years) people or a subset of this population in Canada, Denmark, Norway, Sweden or the UK, from 1970 to 2009.
Intervention
Changes to, or differences in, the generosity and/or eligibility requirements of disability benefits paid to individuals out of the labour market for over 3 months due to health problems or disabilities.
Outcome measures
▶Effect on the probability of being in employment and/or being on disability benefits.
▶Length of time on disability benefits.
▶Length of time off work or not in employment.
We defined disability benefits as ‘state supported income replacement benefits paid to individuals out of the labour market for over 3 months due to health problems or disabilities’. We therefore excluded studies that primarily investigated the effect of economic incentives on short-term sickness absence. We also excluded studies that did not investigate the effect of disability benefit programmes on movement into or out of the labour market; for example, those that only analysed movement between different benefit schemes. We defined eligibility requirements as any criteria or procedures the applicant needs to meet or undergo in order to be eligible for disability benefits.
Flow chart for searches and study selection.
The lead reviewer excluded papers that were considered irrelevant, based on their titles and abstracts. The remaining studies were then evaluated separately by two reviewers against the inclusion criteria and validity assessment. A standardised form was used to collect data on the key characteristics of each study and carry out the validity assessment. When results of multiple models are given in the papers reviewed, the results of the fullest or final model are presented here.
Validity assessment
Econometric studies were the only study type identified through this review. There are no standard tools available for the appraisal of econometric studies.19 After consultation with an expert in the synthesis of econometric studies (N Rice, York University), a simple appraisal framework was developed using core epidemiological principles for assessing validity (appendix 3, available online only).20–22
A total of 28 studies underwent validity assessment, of which 12 were excluded by the process.23–33 The excluded studies had adopted a similar strategy to that criticised by Bound.13 While they used regression models to compare the labour force participation of those with different disability benefit levels, the variation in disability benefit levels were due to the application of the benefit rules rather than a change in those rules resulting from policy decisions. The variation in benefit levels in these studies was therefore determined to a large part by other factors such as age, level of disability, previous earnings or number of dependants, which are used in the benefits system to determine the level of benefits paid. As each of these other factors would also have a direct effect on labour market participation it is unlikely that the regression models in these studies can accurately determine the independent effect of the benefit level. Bound13 14 argues that this analytical strategy significantly overestimates the impact of disability benefits on labour market attachment.
Results
Sixteen studies were included from four countries: eight from Canada, five from the UK, two from Sweden and one from Norway. No studies from Denmark met the inclusion criteria. Nine of these studies investigated variations in benefit generosity only,34–42 three investigated variation in eligibility requirements only,43–45 two investigated variation in both eligibility requirements and benefit generosity as separate parameters in the same model,46 47 and two studies investigated the impact of a single reform that included a combination of changes to eligibility requirements and benefit generosity.48 49 A more detailed description of the policies implemented in these countries, including those that were the focus of papers in this review, is given in appendix 1 (available online only). The main findings are presented for each country separately to take into consideration the country policy and labour market context.
Studies investigating Canadian policy changes
Seven of the eight studies from Canada assessed the impact of changes in the disability component of the Canadian/Quebec pension plan,36 37 39 44–47 and one study investigated the impact of variations in benefits from various sources38 (see table 1). The majority of studies (six/eight) investigated effects on men only and most only reported on people over the age of 45 years (seven/eight). Two of these studies investigated changed eligibility requirements,44 45 four investigated benefit generosity36–39 and two investigated both.46 47
Studies investigating policy changes and differences between jurisdictions in Canada
Of the four papers that investigated changes in the Canadian/Quebec pension plan eligibility requirements, one of these found that there was no association between increased rejection rates (indicating more stringent assessment criteria) and labour market participation.45 Two studies found that some periods of relaxed eligibility were significantly associated with an increase in labour market participation, while others had no significant effect.46 47 The fourth study found that a relaxation of eligibility criteria, which allowed assessors to take into account local labour market conditions in deciding on eligibility, was significantly associated with a decrease in employment.44
Of the six Canadian papers that investigated the effect of differences in benefit replacement rates or benefit levels, four reported that higher benefit levels or replacement rates during the late 1980s and early 1990s were associated with lower male employment.37–39 46 These studies did not sufficiently control for education level38 46 and health status.37 39 One of these studies concluded that although the level of disability benefits did discourage labour force participation, the disincentive effects of low wages had a much greater effect.38 Two studies found that changes in benefit levels had no significant effect.36 47 One of these investigated changes that occurred to benefit levels in 1973,36 and the other investigated the effect of changes in replacement rates between 1983 and 1997.47 This second study found that the effect of changing replacement rates on women was in the opposite direction to that hypothesised by the study: higher replacement rates were associated with higher levels of female labour market participation (p=0.052).
Studies investigating UK policy changes
Five studies of UK benefit policy change were reviewed (table 2). Two of the studies used the British Household Panel Survey to assess the impact of the 1995 Incapacity to Work Act.48 49 This policy included a reduction in the level of benefits paid, particularly for older age groups, and a tightening of eligibility requirements. Disney et al49 did not detect a significant effect from the reforms on the employment of older men with poor health. In contrast, Clasen et al48 concluded that the reforms made transitions from inactivity into employment more likely for 25–49-year-old men (p<0.1) and reduced the flow of older men (aged 50–64 years) from employment into long-term sickness (p<0.1). Neither of these studies controlled for changes in wage levels and Clasen et al48 did not control for changes in health status.
Studies from the UK on benefit changes
Two studies used aggregate time series data to investigate the effect, on labour market participation, of changes in benefit levels and replacement rates between the early 1980s and the end of the 1990s.41 42 Benefit levels had been increasing up to the 1995 reform, which then decreased the benefits paid to older workers considerably. They find that replacement rates41 and benefit levels42 were negatively associated with labour force participation. However, neither study controlled for health status and labour market conditions. When separate age trends were included in the model in the paper by Bell and Smith41 the overall effect was no longer significant. Both studies found that the negative effect of benefit levels on employment was larger for people with no qualifications. Using a model that did not include replacement rates, Faggio and Nickell42 found significant negative effects on labour market participation resulting from falls in regional wages in low level occupations in relation to national wage levels.
Another UK study analysed aggregate data from 1979 to 1984 and found that higher average replacement rates were associated with increasing numbers of people receiving benefits.40 However, the model used did not control for health status or labour market conditions. Given that this period in the UK was one of rapidly rising national unemployment, this would need to be taken into consideration when interpreting the results. In a separate analysis with cross-sectional data they show that unemployment was the dominant factor influencing disability benefit receipt, with higher unemployment levels in an area associated with higher disability benefit receipt.
Studies investigating Swedish policy changes
Two studies from Sweden34 43 investigated changes in sickness and disability insurance policies (see table 3). Hesselius and Persson34 used longitudinal (panel) data to investigate the effects on long-term sickness absence of a 1998 reform to the Swedish national sickness insurance scheme. This reform allowed for additional compensation from collective insurance schemes to be paid on top of national sickness insurance payments after 90 days of sickness absence, resulting in a 12.5% increase in the compensation paid. They found that for people on long-term sickness absence, this reform was associated with an average increase in the duration of sickness absence of 4.7 days (2.8%). In the second study, Karlström et al43 used longitudinal data to investigate a 1997 change in the Swedish disability insurance scheme that abolished favourable treatment for people aged over 60 years. It required applicants to change occupation or residence to find a suitable job, to undertake a more stringent medical test and to engage in rehabilitation. The study did not detect any effect from the reform on the employment of older men (aged 60–64 years).43 They did find, however, that the reform was associated with a decrease in transition from unemployment insurance to disability insurance, a higher transition from employment to sickness insurance, a lower transition from sickness insurance to disability insurance and increased persistence in sickness insurance. In other words, the reform resulted in people shifting between benefits and did not appear to result in increased employment. These two studies were rated through the validity assessment as having the most robust data and analytical approaches.
Studies on Swedish and Norwegian policy changes
Studies investigating policy changes in Norway
One study from Norway was included in the review (see table 3).35 This investigated the effect of changes in the replacement rate in the Norwegian disability insurance scheme between 1971 and 1991. Over this period, average replacement rates rose in the late 1970s and were unchanged or declined slightly during the 1980s.35 The study found no significant relationship between the replacement rate and the numbers of people claiming disability benefits. It concluded that increasing unemployment was more important than increasing benefit levels in explaining rising entry rates into disability benefits during this time period.
Discussion
Our review sought to identify the evidence available from five OECD countries with highly developed social welfare systems, to determine the extent to which variation in the generosity and eligibility requirements of disability benefit programmes affect labour market participation.
First, on the issue of eligibility, there was no clear evidence from these countries that changes in the eligibility requirements of disability benefits had a measurable impact on employment. Of the five studies that specifically addressed this issue, one from Canada found that relaxing eligibility was significantly associated with a decline in employment of older men,44 two papers from Canada found that some periods of relaxed eligibility were associated with a significant increase in employment46 47 and two papers from Canada45 and Sweden43 found no significant effect; importantly this included one of the Swedish papers that were rated as having the highest level of validity. Two papers from the UK assessed the impact of the Incapacity for Work Act, which involved both a reduction in benefit levels and a tightening of assessment approach. These studies gave a mixed picture, one study demonstrated improved employment outcomes,48 while the other did not detect any effect.49 Therefore, we conclude that there is insufficient evidence, and what there is is equivocal, to indicate whether changes in benefit eligibility requirements similar to those studied here will have an impact on the employment of people with disabilities and chronic illness in well developed welfare states.
Second, on the issue of generosity, of the 11 studies that investigated whether the generosity of disability benefits influenced labour market participation, eight reported that benefit levels or benefit replacement rates had a significant negative association with measures of labour market participation.34 37–42 46 Only one of these studies investigated the effect of benefit levels separately on the employment of women and this found no significant effect,47 the others only included men or were on mixed populations. These studies all have substantial validity issues, which we discuss in more detail below. The Swedish study that was assessed as being the most robust did, however, demonstrate a small but significant effect with an increase in benefit of 12.5% associated with a 2.8% increase in the duration of long-term sickness absence.34 While several of the other studies in this review report much larger effects, there is some likelihood that the size and significance of these effects are attributable to other confounding factors and inappropriate statistical methods. We therefore conclude that while it is likely that at some level increased benefit generosity will reduce labour market participation, and that the majority of evidence reviewed here points in that direction, there is insufficient evidence of a high enough quality to determine the extent of that effect.
Limitations of the available evidence
All of these studies rely on ‘natural policy experiments’, arising from governments changing disability benefit schemes over time, or when schemes were administered differently in different jurisdictions, as in Canada. As with other observational studies, we need first to assess whether the size of the effects observed could be attributable to confounding factors or could have occurred by chance.
Conventional economic analysis of welfare systems has been criticised for oversimplifying the relationship between participation in the labour market and financial incentives.8 There are numerous interrelated factors that could influence whether a person developing a health problem will subsequently remain in or return to employment. To determine whether the reported results are actually the result of changes in disability benefits, these other factors need to be taken into account either in the study design or in the analysis. Potential confounding factors in these studies would include changes in labour market conditions, disability and workplace legislation, rehabilitation interventions, as well as differences in individual characteristics such as educational level or health status. However, many of the studies reviewed here had not fully taken this context into account. Three out of the 16 studies reviewed did not control for labour market conditions in their analysis.36 37 41 42 Seven studies used aggregate (ecological) data in which individual characteristics cannot be adequately controlled for.35 39–42 46 47 Even those studies using individual data often lacked sufficient controls for important individual confounders such as educational level, occupation, health status or wages.
It is recognised that these confounding issues can be partly overcome by using a ‘differences in differences’ design and through using fixed effects models with longitudinal (panel) data.50 Nine of the studies in this review34 36 37 42–47 used a difference-in-difference approach. Four of the studies reviewed used longitudinal (panel) data,34 43 48 49 and only two used both.34 43 Fixed effects models will, however, only control for unobserved individual effects if these do not vary over time. Difference-in-difference designs, in which one group has been affected by a policy change while another has not, will still be susceptible to an imbalance in characteristics between these two groups, particularly if this results in different trends over time within subgroups.
The statistical techniques used by many of the studies in this review have been criticised widely in the econometric literature.51–53 In particular, where they have not taken into account serial and spatial correlation in the dependant variable.51 52 Bertrand et al52 demonstrate that this issue could result in difference-in-difference studies reporting a significant effect 45% of the time when in fact there is no effect. Seven of the studies in this review35 37 38 40–42 46 had not adequately taken this issue into account in their analysis. Given the threats to the validity of many of the studies in this review, conclusions are necessarily limited and indicate that there is a lack of evidence from these countries of a high enough quality to determine the extent to which these policies will influence the labour market participation of people with disabilities.
This review did not include all countries that may have met our three policy context criteria. This is a limitation to the review's comprehensiveness, but it did mean that the included countries were examined in greater depth, which was important for appropriate interpretation of the evidence from a policy perspective.
Policy implications
There are various potential reasons why we found no clear evidence that changes in benefit eligibility requirements influenced employment. This may have resulted from the methodological issues discussed, but it is also possible that there is actually no effect from these policies. One possible reason for a lack of effect, suggested by some papers in this review, is that changes in the eligibility structure for one benefit may result in movement into other benefit schemes rather than into the labour market.43 48 For example, Karlstrom et al43 show that changes in the assessment requirements for disability benefits in Sweden resulted in increased persistence of people on sickness and unemployment benefits, but no increase in employment. This indicates that changes to disability benefits need to be coordinated with developments in other welfare benefit schemes. The aim should be to increase employment rather just reducing the number of people on benefits.
While we did not find sufficient evidence of a high enough quality to indicate the extent to which changes in benefit generosity affect employment, several studies indicated that wage levels and the level of unemployment may potentially be more important influences on the employment of people with disabilities. The number of people on disability benefits was found to be strongly positively associated with the level of unemployment on a regional level in the UK and a national level in Norway.35 40 Three studies from the UK and Canada report that the low wages in available jobs were a more important predictor of decreased employment than the level of disability benefits.38 41 42 Other interventions, such as increasing the number of jobs that are accessible to people with disabilities, or subsidising their wages, may be more influential and need to be investigated. Norway, Denmark and Sweden have social democratic welfare-state regimes, characterised by high minimum wages and generous disability benefits compared with the liberal type of regimes of the UK and Canada. It is possible that high minimum wages in the social democratic regimes may give stronger incentives to work, particularly for people with low social status, overriding the disincentive effects of high benefits.
In all five countries included in this review the employment rates of people with a chronic illness or disability decrease steeply with declining socioeconomic status.6 7 Two of the UK studies in this review found that the negative effect of benefit levels on employment was larger for people with no qualifications.41 42 This may reflect the low level of wages for unskilled labour in the UK, or that other factors related to social class are mediating the effect of benefit levels on employment. Further investigation is needed to determine the differential effects of changes in disability benefit systems on both the employment and livelihood of different socioeconomic groups.
Before policy makers consider lowering and/or restricting access to disability benefits, on the assumption that it will increase employment among people with disabilities, they need to weigh up negative as well as positive consequences that could result from this policy. The wider negative consequences have not been assessed, but could potentially include increased poverty for people who already have health problems, possibly exacerbating health inequalities. While changing benefit levels may affect the employment of some claimants at the margins, the consequences of this, in terms of loss of income, affects all claimants. If the employment effects are found to be small and leave more vulnerable groups such as people with mental health problems on reduced benefits, the negative consequences may outweigh the gains made in increasing employment.
Future evaluations of these policies need to determine the extent to which they impact, not only on the employment of people with chronic illness and disabilities, but also on their income, social inclusion and health, as well as any differential impact across health conditions and social groups.
What is already known on this subject
There has been a marked increase in the number of people claiming disability benefits in several OECD countries over recent decades.
Several studies from the USA have concluded that relaxed eligibility requirements and increased generosity of disability benefits are responsible for declines in labour market participation.
There has been no previous systematic review of the evidence on the extent to which these issues have influenced the employment of people with disabilities in well-developed welfare systems.
What this study adds
This study demonstrates that there is a lack of evidence from these countries with well-developed welfare systems that changes in benefit eligibility requirements have an impact on employment. There was more evidence indicating that benefit level was negatively associated with employment, but due to the quality of the evidence reviewed it was not possible to determine the extent of that effect.
This indicates that the impact of policies reducing benefit generosity and narrowing eligibility remain uncertain, with the higher quality study indicating that increased benefit levels reduce labour market participation by a relatively small margin.
This level of uncertainty has important implications for policy makers who need to weigh up the benefits of this policy in terms of employment, against any negative consequences; which could include increased poverty for people who already have health problems.
Acknowledgments
The authors would like to thank Nigel Rice, Professor of Health Economics, the University of York, for his advice on econometric methods and validity assessment.
References
Supplementary materials
Web Only Data jech.2010.111401
Files in this Data Supplement:
Footnotes
Funding SC and MW were funded by the Public Health Research Consortium (PHRC), which is funded by the Department of Health Policy Research Programme. BB was on an attachment with the PHRC as part of the Mersey Deanery Public Health training programme.
Provenance and peer review Not commissioned; externally peer reviewed.