You are here

Article Text

Article menu

Download PDFPDF

Original article
How valid is a short measure of effort–reward imbalance at work? A replication study from Sweden
  1. Constanze Leineweber1,
  2. Natalia Wege2,
  3. Hugo Westerlund1,
  4. Töres Theorell1,
  5. Morten Wahrendorf2,
  6. Johannes Siegrist2
  1. 1Stress Research Institute, Stockholm University, Stockholm, Sweden
  2. 2Department of Medical Sociology, Faculty of Medicine, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany
  1. Correspondence to Constanze Leineweber, Stress Research Institute, Stockholm University, SE-106 91 Stockholm, Sweden; constanze.leineweber{at}stressforskning.su.se

Abstract

Objectives There is an urgent need for validated measures of health-adverse psychosocial work environments. We tested the validity of a newly developed short version of the original questionnaire measuring effort–reward imbalance at work (ERI).

Methods The study sample comprised working men and women (n=4771) participating in the Swedish Longitudinal Occupational Survey of Health (SLOSH), a nationally representative longitudinal cohort study, in 2006 and 2008. Structural equation modelling was applied to test factorial validity, using the ERI scales. Furthermore, criterion validity was explored with two prospectively assessed health indicators, poor self-reported health and depressive symptoms. Results are based on logistic and linear regression analyses, with appropriate confounder control.

Results The short version of the ERI questionnaire (16 items) provides satisfactory psychometric properties (internal consistency of scales, confirmatory factor analysis with a good model fit of the data with the theoretical structure). All scales, and the effort–reward ratio, were prospectively associated with an increased risk of poor general self-rated health and depressive symptoms, indicating satisfactory criterion validity.

Conclusion This short version of the ERI questionnaire provides a psychometrically useful tool for epidemiological studies focused on the health-adverse effects of work and employment in the context of a globalised economy.

  • Effort reward imbalance
  • psychometric properties
  • short version
  • work stress
  • epidemiology
  • stress
  • longitudinal studies

https://doi.org/10.1136/oem.2009.050930

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    What this paper adds

    • Recently the psychometric properties of a short version of the effort–reward imbalance questionnaire have been tested in a cross-sectional sample.

    • In this article we show that this short measure of effort–reward imbalance at work also demonstrates satisfactory psychometric properties in longitudinal data.

    • Employees reporting high effort at work that are not reciprocated by adequate rewards (salary, promotion, job security, esteem) are at increased risk of poor self-rated health and depressive symptoms at follow-up.

    • Standardised valid and economic assessment tools, such as this short measure, are useful to guide the activities of worksite health promotion programs.

    Introduction

    In the context of economic globalisation, work pressure is increasing in many sectors of the labour market, often in combination with inappropriately low wages and reduced job security.1 The theoretical model of effort–reward imbalance at work was developed to identify health-adverse effects produced by a stressful psychosocial work environment.2 The model claims that recurrent lack of reciprocity between efforts made at work by employees and rewards received in turn from employers (‘high cost-low gain’) elicits strong negative emotions with unfavourable long-term effects on health. Rewards concern employees' salary, their promotion prospects including job security and esteem related to their performance. Available evidence indicates an increased risk of stress-related physical and mental disorders among people who experience effort–reward imbalance at work.3–6

    To measure this model of a health-adverse psychosocial work environment, a standardised self-report questionnaire was developed.7 The use of self-report measures of stressful work has been criticised on methodological grounds, and quality criteria required for robust evidence were proposed to meet these challenges.8–10 Criteria include the separate measurement of exposure and outcome (eg, in terms of time, that is, a prospective study design), the demonstration of appropriate psychometric properties of the exposure measure, including test of the factorial structure of the scales measuring the construct, and the replication of findings in independent samples. While several of these requirements were already met in the original questionnaire measuring effort–reward imbalance at work, a shorter version of this questionnaire was developed with the intention of providing an economic measure for inclusion in large-scale epidemiological investigations.7 11 12

    In this report we set out to test the validity of this short questionnaire by replicating its psychometric properties, including its construct validity, in an independent sample and by analysing its contribution to the prediction of two prospectively assessed indicators of health.1 2 Moreover, while the first test of this short measure was conducted in a representative sample of German-speaking people, this replication study was realised in a representative sample of Swedish employees. Analysing self-report measures in different languages is important not only in terms of their wider application, but also in terms of the cross-cultural significance of the particular dimensions of stressful experience at work that are inherent in the theoretical construct.

    Methods

    Study population

    This report is based on the first two data waves of SLOSH (Swedish Longitudinal Occupational Survey of Health), a nationally representative longitudinal cohort survey.13 14 It began in 2006, when the Stress Research Institute commissioned a follow-up of the Swedish Work Environment Survey (SWES) which was carried out in 2003. SWES, which is conducted by Statistics Sweden, is a biennial self-completed questionnaire survey on the physical and psychosocial work environment, answered by a stratified selection of respondents to the Labor Force Survey (LFS). Respondents are aged 16–64 years. Participants of SLOSH are followed biennially by means of two versions of a postal questionnaire, one for currently working respondents and one for respondents who are currently not working (eg, retired or unemployed). Compared with SWES, SLOSH contains more detailed questions about work environment and health. The questionnaire for workers is divided into three parts. The first part focuses on several aspects of the physical and psychosocial work environment. This part includes the short version of the effort–reward imbalance (ERI) questionnaire (see below). The second part of the SLOSH questionnaire covers health-related issues including self-rated health and depressive symptoms. The third part concerns socio-demographic and economic conditions, including social factors outside work (family structure, housing) and more general aspects of life satisfaction. The first wave of SLOSH was launched in 2006, and the second wave was conducted in 2008. A total of 5985 individuals (65% of the eligible SWES participants) responded to the first wave, of whom 5141 used the questionnaire for employed people and 844 the questionnaire for those not in the labour force. The second wave sample was much larger as it included participants from the SWES conducted in 2005. The sample analysed in the present report is restricted to men and women with complete data from both waves and who were working at least 30% of a 40 h week at baseline in 2006. This sample comprises 4771 respondents. The study has been approved by the Regional Research Ethics Board in Stockholm.

    Measurement

    Effort–reward imbalance

    Effort and reward, and over-commitment were measured by the short version of the ERI questionnaire where effort is measured by three questions, reward by seven questions and over-commitment by six questions.7 As in the original version, reward consists of three components: ‘esteem’, represented by two questions, ‘job promotion’, represented by three questions, and ‘job security’, represented by two questions. Effort and reward were measured by a five-option response scale ranging from ‘disagree’, via ‘agree, and I am somewhat distressed’, to ‘agree, and I am very distressed’. Regarding ‘over-commitment’, respondents were asked to answer on a scale from 1 (‘strongly disagree’) to 4 (‘strongly agree’). Sum scores of these ratings were calculated with appropriate recoding, such that high scores reflect high effort, high reward and high over-commitment . Thus, the range of the effort scale is 3–15, of the reward scale 7–35 and of the over-commitment scale 6–24. Additionally, an effort–reward ratio is calculated by dividing effort by reward, multiplied by the ratio of the number of items (three out of seven). For regression analyses, the effort–reward ratio and all scales were divided into quartiles and treated as categorical variables.

    Self-rated health

    General self-rated health was measured in 2006 and in 2008 by the one-item question: ‘How would you rate your general state of health?’. Respondents answered on a scale from 1 to 5 ranging from ‘very good’ to ‘very poor’. Self-rated health was dichotomised into good/average (‘very good’ to ‘neither good nor bad’) and poor (‘quite poor’ and ‘poor’). ‘Poor health’ in 2008 was considered an outcome criterion in this report. This measure was shown to predict morbidity and mortality in a variety of epidemiological investigations.15 16

    Depressive symptoms

    In both waves, depressive symptoms were measured with a brief subscale from the Hopkins Symptom Checklist (SCL-90).17 The scale measures 1-week prevalence and includes six items covering the depressive core symptoms: ‘feeling blue’, ‘feeling no interest in things’, ‘feeling lethargy or low in energy’, ‘worrying too much about things’, ‘blaming yourself for things’ and ‘feeling everything is an effort’. Response options range from ‘not at all’ to ‘a great deal’. A sum score of depressive symptoms ranging from 6 to 30 was calculated. Depressive symptoms assessed in 2008 were used as the second criterion variable. In this case, continuous data were analysed as no validated cut-off point has yet been established.

    Covariates

    Sex and age at inclusion were derived from the 10-digit personal identification numbers corresponding to US social security numbers. Education was self-reported and measured in 2006. Gross salary in 2003 was derived from the Income and Tax Register.

    Statistical analysis

    Factorial validity was tested with structural equation modelling. We used the χ2 test, the goodness-of-fit index (GFI), which indicates the amount of variance and covariance explained by the model (values over 0.90 indicating acceptable fit), the adjusted goodness-of-fit index (AGFI, which adjusts the GFI for degrees of freedom in the model with values above 0.90 indicating a good fit to the data) and the RMSEA criterion.18–20 The comparative fit index (CFI) was used as a measure of comparative fit with values exceeding 0.90 indicating a good fit to the data.18 Parsimonious fit was measured by consistent Akaike information criterion (CAIC). Testing was carried out in three steps. First, a one-factor-model was estimated where all 16 items loaded on the same underlying dimension (null model). In a second step, a model was estimated with the three first-order factors ‘effort’, ‘reward’ and ‘over-commitment’ loading on the second-order factor ‘effort–reward imbalance’ (model 1). Third, a second-order model representing the theoretical model, with reward represented by three components was estimated (model 2) (figure 1). The associations between exposure variables and health outcomes were examined by logistic (self-rated health) and linear (depression) regression analyses. In a first step, we calculated an unadjusted model, in a second step we adjusted for age and gender, and in a third step we additionally adjusted for education and income. In the fully adjusted model we also adjusted for self-rated health or depressive symptoms at baseline, respectively. We used SPSS for Windows V.15.0 and AMOS 7.0 for the statistical analyses.

    Figure 1
    Figure 1

    Competitive models of confirmatory factor analysis testing the theoretical construct underlying the shortened Effort-Reward Imbalance Questionnaire (Models 1 and 2, see text).

    Results

    Sample descriptions are shown in table 1. The participants were on average 47 years old in 2006 and most of them were highly educated. Slightly more women than men participated in the study. Most participants rated their health as good in both 2006 and 2008.

    View this table:
    Table 1

    Characteristics of the sample analysed (n=4771)

    Internal consistency

    Item-total correlation and Cronbach's α are presented in table 2. All Cronbach's α coefficients are equal to or higher than 0.80, indicating a satisfying internal consistency. Item-total correlations varied between 0.55 (0.42 for corrected item-total correlation) and 0.86 (0.78 for corrected item-total correlation) and were all above the threshold of 0.30.21

    View this table:
    Table 2

    Mean, SD, Cronbach's α and item-total correlation coefficients (uncorrected/corrected) of the scales of the Swedish version of the shortened effort–reward imbalance questionnaire

    Factorial validity

    The factorial validity of the ERI scales was tested with confirmatory factor analysis. Table 3 shows the summary of the fit-indices for the three competitive models of the confirmatory factor analysis. The first-order model fits the data better than the null model. However, only the second-order model reached an acceptable level of fit (GFI>0.90, AFGI>0.90, RMSEA<0.08). CFI and CAIC as measures of comparative and parsimonious fit favour the second-order model (figure 1).

    View this table:
    Table 3

    Summary of goodness-of-fit for the models for the three competitive models of confirmatory factor analyses

    Criterion validity

    We tested health as two aspects of criterion validity of the short ERI measure, that is self-perceived general health and depressive symptoms. Thus, we assume that employees who scored high on the scales of the construct were at elevated risk of experiencing poor self-rated health and more depressive symptoms.

    Regarding general self-rated health, we found a clear dose–response effect for all scales and for the ERI construct (table 4). Participants who experienced the highest ERI, the highest efforts, the lowest rewards, and were highly overcommitted in 2006 showed a heightened risk of poor self-rated health in 2008. This holds true after adjustment for age, sex, education and gross income. Controlling additionally for self-rated general health at baseline attenuated the association somewhat. However, participants in the highest quartiles still exhibited more than double the risk for poor self-rated general health compared to those in the lowest quartile. For depressive symptoms a similar graded association was observed, with the least favourable situation yielding the highest estimates. In unadjusted linear regression analyses, all predictors were significantly associated with elevated scores of depressive symptoms 2 years later (table 5). Additional adjusting for age and sex, as well as for age, sex, education and gross income, did not change the results substantially in comparison with the unadjusted model (see online table W1). After adjusting for age, sex, education, gross income and depressive symptoms at baseline, the estimates decreased substantially. Yet, in the final statistical model with full adjustment, scoring in the top two quartiles of all ERI scales and of the ratio was still associated with significantly elevated scores of depressive symptoms at follow-up.

    View this table:
    Table 4

    Association between work stress in 2006 and poor self-rated health in 2008

    View this table:
    Table 5

    Association between work stress in 2006 and depressive symptoms in 2008

    Discussion

    The findings of this study confirm the previously documented psychometric properties of an abbreviated version of the original questionnaire measuring effort–reward imbalance at work in an independent large, representative sample of the Swedish working population. Data show satisfactory internal consistency of the scales, and confirmatory factor analyses resulted in a good model fit of the data with the most appropriate representation of the theoretical structure of the model. The goodness-of-fit criteria for model 2 were very similar to those observed in the German study providing the first test of this short measure.12 Moreover, comparison of psychometric properties of the shorter version with the original questionnaire reveals the high degree of comparability, supporting the theoretical construct of the effort–reward imbalance model.7 12 Thus, it seems possible to reproduce a relatively complex theoretical notion (model 2 in figure 1) using a standardised questionnaire composed of no more than 16 items.

    Furthermore, all scales and the effort–reward ratio were found to be prospectively associated with an increased risk of poor general self-rated health and depressive symptoms, supporting appropriate criterion validity of the measurement. Results are in accordance with those obtained from prospective observational cohort studies using either the original ERI questionnaire or a proxy measure where risks of depressive symptoms and poor self-rated health were elevated among those reporting effort–reward imbalance at baseline.22–26

    The good comparability of the psychometric properties of the scales and of their structure observed in two large, representative groups of working populations in Germany and in Sweden where selection effects are unlikely, is considered a particular strength of this newly designed short measure. An additional strength is obvious from the fact that the short measure is significantly associated with two indicators of prospectively assessed health. Although we cannot rule out a possible reporting bias due to self-reported exposure and outcome data, previous reports demonstrated that adjustment for personal response style (eg, negative affectivity) does not invalidate the observed associations of effort–reward imbalance at work with reduced health.25 27 28

    Several limitations are obvious from this report. First, we provide only overall results, without specifying them according to occupational categories, socio-economic subgroups, duration of employment, etc. However, analyses on different occupational groups, that is blue-collar workers, white-collar workers and managers, showed that the short version of the ERI and its subscales were associated with a significantly higher risk of poor self-rated health among both blue- and white-collar workers, albeit with different strengths. No significant results were found for managers, probably due to lack of power (n=523). These results may indicate that reward is more strongly related to self-rated health among white-collar workers than among blue-collar workers, or that the operationalisation of the ERI in this version is less well adapted to the conditions of blue-collar workers. Furthermore, we ran the analyses for those subjects who stayed with the same employer from 2003 to 2007 (n=3257, 68.3%). These analyses showed even more pronounced effects of the ERI and its subscales in this group. Among those who changed employer the effects were less pronounced, possibly indicating that the effects of effort–reward are accumulated over time. Second, we collected data on the exposure only once, at baseline, and therefore have no information about possible impact on health produced by changes between the first and second measurements. Third, depressive symptoms were measured by a short scale of the Hopkins Symptom Checklist which has so far not been formally validated. However, this depression subscale is one of the most commonly used depression scales and has been shown to function well as a screening instrument in a Swedish setting.29

    In conclusion, despite these limitations, results of this study indicate that a short standardised questionnaire measuring an imbalance between high effort spent and low reward received at work provides a psychometrically useful tool for epidemiological studies interested in health-adverse effects of work and employment in the context of a globalised economy.

    References

    1. ↵
      1. Spiegel JM,
      2. Labonte R,
      3. Ostry A
      . Understanding 'globalization' as a determinant of health determinants: a critical perspective. Int J Occup Environ Health 2004;10:360–7.
      OpenUrlPubMedWeb of Science
    2. ↵
      1. Siegrist J
      . Adverse health effects on high-effort/low reward conditions. J Occup Health Psychol 1996;1:27–41.
      OpenUrlCrossRefPubMed
    3. ↵
      1. Kivimäki M,
      2. Virtanen M,
      3. Elovainio M,
      4. et al
      . Work stress in the etiology of coronary heart disease - a meta-analysis. Occup Environ Med 2006;32:431–42.
      OpenUrl
    4. ↵
      1. Cartwright S,
      2. Cooper CL
      1. Siegrist J
      . Job control and reward. In: Cartwright S, Cooper CL, eds. Handbook of organizational well being. Oxford: Oxford University Press, 2008:109–32.
    5. ↵
      1. Stansfeld S,
      2. Candy B
      . Psychosocial work environment and mental health—a meta-analytic review. Scand J Work Environ Health 2006;32:443–62.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Tsutsumi A,
      2. Kawakami N
      . A review of empirical studies on the model of effort-reward imbalance at work: reducing occupational stress by implementing a new theory. Soc Sci Med 2004;59:2335–59.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Siegrist J,
      2. Starke D,
      3. Chandola T,
      4. et al
      . The measurement of effort-reward imbalance at work: European comparisons. Soc Sci Med 2004;58:1483–99.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Kasl SV
      . The influence of work environment on cardiovascular health: A historical, conceptual and methodological perspective. J Occup Health Psychol 1996;1:42–56.
      OpenUrlCrossRefPubMed
    9. ↵
      1. van Vegchel N
      . Two models at work. Utrecht: Utrecht University, 2005.
    10. ↵
      1. Zapf D,
      2. Dormann C,
      3. Frese M
      . Longitudinal studies on organizational stress research: a review of the literature with reference to methodological issues. J Occup Health Psychol 1996;1:145–69.
      OpenUrlCrossRefPubMed
    11. ↵
      1. de Jonge J,
      2. van der Linden S,
      3. Schaufeli WB,
      4. et al
      . Factorial invariance and stability of the effort-reward imbalance scales: a longitudinal analysis of two samples with different time lags. Int J Behav Med 2008;15:62–72.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Siegrist J,
      2. Wege N,
      3. Pühlhofer F,
      4. et al
      . A short generic measure of work stress in the era of globalization: effort-reward imbalance. Int Arch Occup Environ Health 2009;82:1005–13.
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Kinsten A,
      2. Magnusson Hanson L,
      3. Hyde M,
      4. et al
      . SLOSH - Swedish Longitudinal Occupational Survey of Health—a nationally representative psychosocial survey of the Swedish working population. Stress Research Report. Stockholm: Stress Research Institute, 2007.
    14. ↵
      1. Magnusson Hanson L,
      2. Theorell T,
      3. Oxenstierna G,
      4. et al
      . Demand, control and social climate as predictors of emotional exhaustion symptoms in working Swedish men and women. Scand J Public Health 2008;36:737–43.
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Benyamini Y,
      2. Idler EL
      . Community studies reporting association between self-rated health and mortality: additional studies, 1995 to 1999. Research Aging 1999;21:392–401.
      OpenUrlFREE Full Text
    16. ↵
      1. DeSalvo KB,
      2. Bloser N,
      3. Reynolds K,
      4. et al
      . Mortality prediction with a single general self-rated health question. A meta-analysis. J Gen Intern Med 2006;21:267–75.
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Magnusson Hanson LL,
      2. Theorell T,
      3. Bech P,
      4. et al
      . Psychosocial working conditions and depressive symptoms among Swedish employees. Int Arch Occup Environ Health 2009;82:951–60.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Kelloway EK
      . Using LISREL for structural equation modeling: A researcher's guide. Thousand Oaks, CA: Sage, 1998.
    19. ↵
      1. Kline R
      . Principles and practice of structural equation modelling. New York: Guilford Press, 2005.
    20. ↵
      1. Kelloway EK
      . Structural equation modelling in perspective. Journal of Organizational Behavior 1995;16:215–24.
      OpenUrlCrossRefWeb of Science
    21. ↵
      1. Nunnally JC,
      2. Bernstein IH
      . Psychometric theory. 3rd edn. New York: McGraw-Hill, 1994.
    22. ↵
      1. Buddeberg-Fischer B,
      2. Stamm M,
      3. Buddeberg C,
      4. et al
      . Work stress, health and satisfaction of life in young doctors: results of a longitudinal study in Switzerland (in German). Dtsch Med Wochenschr 2008;133:2441–7.
      OpenUrlCrossRefPubMed
    23. ↵
      1. Godin I,
      2. Kittel F
      . Differential economic stability and psychosocial stress at work: associations with psychosomatic complaints and absenteeism. Soc Sci Med 2004;58:1543–53.
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Kivimäki M,
      2. Vahtera J,
      3. Elovainio M,
      4. et al
      . Effort-reward imbalance, procedural injustice and relational injustice as psychosocial predictors of health: complementary or redundant models? Occup Environ Med 2007;64:659–65.
      OpenUrlAbstract/FREE Full Text
    25. ↵
      1. Stansfeld S,
      2. Fuhrer R,
      3. Shipley MJ,
      4. et al
      . Work characteristics predict psychiatric disorder: prospective results from the Whitehall II Study. Occup Environ Med 1999;56:302–7.
      OpenUrlAbstract/FREE Full Text
    26. ↵
      1. Niedhammer I,
      2. Tek ML,
      3. Starke D,
      4. et al
      . Effort-reward imbalance model and self reported health: cross-sectional and prospective findings from the GAZEL cohort. Soc Sci Med 2004;58:1531–41.
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      1. Bosma H,
      2. Peter R,
      3. Siegrist J,
      4. et al
      . Two alternative job stress models and the risk of coronary heart dsease. Am J Public Health 1998;88:68–74.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Joksimovic L,
      2. Starke D,
      3. von dem Knesebeck O,
      4. et al
      . Perceived work stress, overcommitment, and self-reported musculo-skeletal pain: a cross-sectional investigation. Int J Behav Med 2002;45:61–71.
      OpenUrl
    29. ↵
      1. Fridell M,
      2. Cesarec Z,
      3. Johansson M,
      4. et al
      . SCL-90. Svensk normering, standardisering och validering av symtomskalan [SCL-90. Swedish standardization and validation of the symptom scale]. SiS följer upp och utvecklar: Statens institutionsstyrelse SiS, 2002:9–66.

    Footnotes

    • Funding CL, HW and TT were funded by the Swedish Council for Working Life and Social Research (FAS, grant #2004–2021). The SLOSH study is funded by the Swedish Council for Working Life and Social Research (FAS, grant #2005-0734). JS, NW and MW were funded by the German Research Foundation (Project SI 236-10/1).

    • Competing interests None.

    • Ethics approval This study was conducted with the approval of the Regional Research Ethics Board in Stockholm.

    • Provenance and peer review Not commissioned; externally peer reviewed.