Elsevier

Social Science & Medicine

Volume 62, Issue 4, February 2006, Pages 941-953
Social Science & Medicine

Psychometric and cognitive validation of a social capital measurement tool in Peru and Vietnam

https://doi.org/10.1016/j.socscimed.2005.06.050Get rights and content

Abstract

Social capital is a relatively new concept which has attracted significant attention in recent years. No consensus has yet been reached on how to measure social capital, resulting in a large number of different tools available. While psychometric validation methods such as factor analysis have been used by a few studies to assess the internal validity of some tools, these techniques rely on data already collected by the tool and are therefore not capable of eliciting what the questions are actually measuring.

The Young Lives (YL) study includes quantitative measures of caregiver's social capital in four countries (Vietnam, Peru, Ethiopia, and India) using a short version of the Adapted Social Capital Assessment Tool (SASCAT). A range of different psychometric methods including factor analysis were used to evaluate the construct validity of SASCAT in Peru and Vietnam. In addition, qualitative cognitive interviews with 20 respondents from Peru and 24 respondents from Vietnam were conducted to explore what each question is actually measuring.

We argue that psychometric validation techniques alone are not sufficient to adequately validate multi-faceted social capital tools for use in different cultural settings. Psychometric techniques show SASCAT to be a valid tool reflecting known constructs and displaying postulated links with other variables. However, results from the cognitive interviews present a more mixed picture with some questions being appropriately interpreted by respondents, and others displaying significant differences between what the researchers intended them to measure and what they actually do.

Using evidence from a range of methods of assessing validity has enabled the modification of an existing instrument into a valid and low cost tool designed to measure social capital within larger surveys in Peru and Vietnam, with the potential for use in other developing countries following local piloting and cultural adaptation of the tool.

Introduction

Social capital is a way of describing social relationships within societies or groups of people. It is a relatively new concept which has attracted significant attention in the field of health research in recent years. Research has shown social capital to be associated with a wide range of health outcomes including mortality (Kawachi, Kennedy, Lochner, & Prothrow-Smith, 1997; Skrabski, Kopp, & Kwachi, 2003), self-reported health status (Kawachi, Kennedy, & Glass, 1999; Veenstra, 2000), and mental health (De Silva, McKenzie, Huttly, & Harpham, 2005).

Within the health field, consensus is slowly being reached about the definition of social capital as referring to “social networks and their associated norms of reciprocity” (Putnam, 2004). Social capital comprises the features of social organisation and integration that facilitate co-operation for mutual benefit. This includes the quantity and quality of formal and informal social interactions (often called associational life), civic participation, norms of reciprocity and trust in others. Social capital is multi-dimensional and includes structural (quantity of social relationships) and cognitive (quality of social relationships) components (Bain & Hicks, 1998). Structural and cognitive social capital can refer to linkages and perceptions in relation to people who are similar to each other such as others in ones own community or people of the same socio-economic status (called bonding social capital), or to people who are different, such as people outside ones community or with a different social identity (called bridging social capital). Social capital can also occur through formal institutions such as between a community and local government structures, and this is termed linking social capital (Szreter & Woolcock, 2004).

There has been much debate as to whether social capital should be considered the property of individuals or of groups of people (an ecological construct) (Kawachi, Kim, Coutts, & Subramanian, 2004). Individual social capital is most commonly measured by asking individuals about their participation in social relationships (structural social capital) and their perceptions of the quality of those relationships (cognitive social capital), while ecological social capital is most often measured by aggregating the responses of individuals to the community level. In the health field, despite the emphasis on the importance of ecological measures by some commentators (see for example Kawachi et al., 2004; McKenzie, Whitley, & Weich, 2002), the majority of research has explored the association between individual measures of social capital and health (De Silva et al., 2005).

Despite this theoretical development, significant questions surrounding the measurement of social capital remain, in particular how to translate the different theoretical components of social capital into valid and measurable constructs (Kawachi et al., 2004). This has resulted in the development of a wide range of tools to measure social capital (for example Buckner, 1988; Grootaert & van Bastelaer, 2002; Harpham, Grant, & Thomas, 2002; Hean, Cowley, Forbes, Griffiths, & Murrells, 2003; Narayan & Cassidy, 2001; Yang, Yang, Shih, & Kawachi, 2002). Yet Van Deth's (Van Deth, 2003) plea that “assessing the validity of each measure of social capital in different settings (both cross-cultural and longitudinal) should be standard practice among empirical researchers in this area” has not been heeded. A search of the literature found only eleven studies attempting some validation of social capital tools, despite there being well over 150 studies cited in Medline examining the association between social capital and health (Kawachi et al., 2004), and many hundreds more exploring the relationship between social capital and non-health related outcomes (Halpern, 2004, cited in Putnam, 2004). The problem, partly due to differences in the conceptualisation of social capital, is that researchers have not settled on one tool to measure social capital and then validated it over a number of years. Instead they have developed new tools, sometimes without an accompanying assessment of the tools’ validity. This compounds the situation as with so few validity studies the evidence is not available to distinguish between existing tools.

Of the eleven studies that did conduct some validation of their social capital tool, nine used psychometric validation such as factor analysis to assess internal validity (Hean et al., 2003; Li, Pickles, & Savage, 2003; Narayan & Cassidy, 2001; O’Brien, Burdsal, & Molgaard, 2004; Onyx & Bullen, 2000; Robinson & Wilkinson, 1995; Stone & Hughes, 2002; Yang et al., 2002; Young, Russell, & Powers 2004). All of these studies found the tools they validated were able to distinguish between the different theoretical constructs of social capital, and therefore to have acceptable discriminant validity. However, in a field where no gold standard measure is available to assess concurrent validity, a broader approach to validation is necessary.

As Bowden et al. argue (Bowden, Fox-Rushby, Nyandieka, & Wanjau, 2002), psychometric validation does not contain any analysis from the respondents’ viewpoint, a perspective which is vital in order to understand how respondents interpret the questions and therefore what the tool is actually measuring. Two of the eleven studies did use cognitive validation techniques (Boreham, 1999; Earthy, Maltby, Arber, & Cooper, 2000), but neither was set in the developing world. The results from these studies highlight the importance of using qualitative methods of validation in addition to more standard quantitative approaches, with significant differences reported between what the researchers believed they were asking, and the way in which the respondents interpreted the questions.

While generic tools are often used to measure social capital in different cultural settings, Szreter and Woolcock (2004) argue that social capital is a product of the prior history of political, constitutional and ideological developments in any given setting. As such, it is important to validate a generic tool in each cultural setting in which it is to be applied. The organisations and social networks which are important for structural social capital may differ between different cultures, while culture may affect perceptions of social relationships (cognitive social capital) for example notions of trust. This means that the same question may be interpreted differently in different cultural settings, and culturally specific questions may need to be asked in order to capture the range of social capital available. This paper aims to validate an existing tool to measure social capital using psychometric and cognitive techniques in two different cultural settings (Peru and Vietnam) in order that the association between social capital and health can be more accurately estimated.

Section snippets

Instrument

A relatively short instrument to measure the social capital of adults in a quantitative manner has been formed from a longer instrument developed by a team from the World Bank (Krishna & Shrader, 2000). The instrument is intended for use in surveys where social capital is just one element of a broader study. The resulting Adapted Social Capital Assessment Tool (A-SCAT) (Harpham et al., 2002) has been used in Colombia (Harpham, Grant, & Rodrigues, 2004) and Sub-Saharan Africa (Thomas, 2003). It

Methods and results: psychometric assessment of validity

In order to provide a comprehensive assessment of the validity of SASCAT, each type of construct validity outlined in the schema developed by Trochim (2000) was assessed. Methods and results are presented separately for each aspect of construct validity.

Methods

The cognitive validation comprised two stages. Firstly, an in-depth interview was conducted by MDS with the designer of the SASCAT (TH) to establish what each question was intended to measure. In-depth cognitive interviews were then conducted with 20 Peruvian and 24 Vietnamese respondents, purposively selected to be representative of the sample used for the YL survey (the full methods of this study are presented elsewhere (De Silva et al., in press; Tuan, Harpham, Huong et al., in press).

In

Discussion of results

This paper argues that cognitive validation is a valuable addition to psychometric techniques when validating complex tools for use in different cultural settings. Psychometric techniques to explore discriminant and predictive validity show SASCAT to be a valid tool reflecting known constructs and displaying postulated links with other variables. However, these techniques rely on data already collected by the tool and are therefore not capable of eliciting what the questions are actually

Acknowledgements

We wish to thank Nguyen Thu Huong, Van Thi Thuy Huong, Tran Thap Long, Nguyen Thi Van Ha for help conducting the Vietnamese validation study, and Dr. Javier Escobal, for comments on an earlier version of this document. We also thank Dr. Claudio Lanata and all the field teams in Peru and Vietnam who made the collection of the Young Lives data possible. Thank you also to all the respondents of our qualitative interviews who gave their time and opinions so generously, and to the two anonymous

References (38)

  • M.J. De Silva et al.

    Social capital and mental illness: A systematic review

    Journal of Epidemiology and Community Health

    (2005)
  • L.M.D. deUlzurrun

    Associational membership and social capital in comparative perspective: A note on the problems of measurement

    Politics and Society

    (2002)
  • S. Earthy et al.

    The use of cognitive interviewing to develop questions on social capital for the 2000/1 General Household Survey

    Survey Methodology Bulletin

    (2000)
  • Escobal, J., Lanata, C., Madrid, S., Penny, M., Saavedra, J., Suárez, P., et al. (2003). Young Lives preliminary...
  • C. Grootaert et al.

    Understanding and measuring social capital: A multidisciplinary tool for practitioners

    (2002)
  • D. Halpern

    Social Capital

    (2004)
  • T. Harpham et al.

    Measuring social capital within health surveys: Key issues

    Health Policy and Planning

    (2002)
  • S. Hean et al.

    An examination of the potential to identify an instrument reflecting measurable attributes of social capital—Final report

    (2003)
  • G. Kaplan

    What's wrong with social epidemiology, and how can we make it better?

    Epidemiologic Reviews

    (2004)
  • Cited by (151)

    • The relevance of social capital and sense of coherence for mental health of refugees

      2022, SSM - Population Health
      Citation Excerpt :

      For the same reason and because of low endorsement (7.9%) of the first response category, cognitive social capital was dichotomized into low (scored as 0–2) and high (scored as 3–4). This dichotomy classification followed Flores et al. (2014), as initially proposed by De Silva et al. (2006). Statistical analyses were performed using IBM SPSS Statistics for Windows, Version 27.

    View all citing articles on Scopus
    View full text