Article Text

PDF

Design of a Community Ownership and Preparedness Index: using data to inform the capacity development of community-based groups
  1. Tom Thomas1,
  2. Pradeep Narayanan1,
  3. Tisha Wheeler2,
  4. Usha Kiran3,
  5. M J Joseph1,
  6. T V Ramanathan4
  1. 1Praxis Institute for Participatory Practices, New Delhi, India
  2. 2Futures Group, Washington DC, USA (formerly with the Bill & Melinda Gates Foundation, India Office, New Delhi, India)
  3. 3India Office, Global Health Program, Bill & Melinda Gates Foundation, New Delhi, India
  4. 4Department of Statistics, University of Pune, Pune, India
  1. Correspondence to Pradeep Narayanan, Research and Consultancies, Praxis Institute for Participatory Practices, C-75, South Extension, Part II, New Delhi-110049, India; pradeepn{at}praxisindia.org

Abstract

Background Community mobilisation is an important component of a participatory approach to health and development interventions. However, it is challenging to define, measure and assess community participation and ownership of a programme, especially at scale.

Methods An iterative cross-sectional survey was designed for implementation across a representative sample of community-based groups, using a weighted index that captured both qualitative and quantitative data in a standardised form. These data were aggregated at the level of individual groups, as well as state-wide or across the whole programme. Community participation in the survey is a primary feature of the methodology and was integral to the process of designing the index and administering the survey.

Results The survey provided programme management and communities with objective tools for monitoring community mobilisation across a large-scale and complex intervention covering 32 districts in India. The implementation of the survey engaged communities in an open discussion of their goals and capabilities and helped them to challenge the power dynamics between themselves and other stakeholders.

Conclusions It is possible to translate the theoretical premises of participatory development into a tool that both measures and fosters meaningful participation. The active participation of community members in the collection and analysis of data on their mobilisation suggests that monitoring of participation can be undertaken to inform a scaled-up programme and can be a useful intervention in its own right.

  • Social science
  • community development
  • capacity development
  • HIV/AIDS
  • public health policy
  • anthropology
  • AIDS
  • education
  • socio-economic
  • social inequalities
  • social capital
  • sexually transmitted disease

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

Statistics from Altmetric.com

Introduction

Community mobilisation, when implemented effectively and integrated into health and development interventions, is acknowledged as an approach that can improve outcomes and make them more sustainable, and achieve broader goals of addressing poverty and fostering well-being.1 It has also been recognised as an approach to addressing vulnerability to HIV, which stems from social, economic or legal circumstances that increase susceptibility to infection, deter individuals from seeking essential prevention services or enhance the likelihood of engaging in unsafe behaviour.2 Many argue that an HIV prevention intervention will be more effective and more sustainable if it develops the capacity of those most at risk of HIV/AIDS to tackle issues of discrimination, stigma, exclusion and powerlessness.3

Community-based groups (CBGs) are a frequent feature of community mobilisation.4 In HIV/AIDS programmes, CBGs and networks are seen as a vehicle to strengthen demand for services and manage programmatic activities.5 However, apart from the delivery of public health, community mobilisation has also influenced many important HIV/AIDS policy decisions, and marginalised groups affected by HIV have found multiple incentives for coming together.6 Such incentives, also reflected in social movements globally, include: generating and accessing resources;7 addressing the rights of group members;8 sharing a collective identity, especially in their common experience of injustice;9 and advancing a political agenda.10 The mobilisation and collective action of Indian sex workers through the Sonagachi project in Kolkata has drawn widespread interest,11 and a growing literature from across India suggests that the country may continue to influence community mobilisation as a structural approach to HIV/AIDS.12–15

However, community mobilisation is often defined imprecisely and neither measured nor monitored systematically. This makes it hard to determine progress and to guide capacity development and participatory learning and planning processes at scale. Measurement of community mobilisation has received notable attention in academic and research settings,16 ,17 and small-scale projects have produced compelling results,18 ,19 but there remains a gap in the understanding of how to apply measurement of community participation in scaled-up operational settings. In some scaled-up processes where community or civil society forums are convened to influence national project approaches—for example, in Poverty Reduction Strategy Papers (PRSPs)—there has been widespread criticism of the quality of participation.20 PRSPs are the national planning mechanism required of developing countries by the World Bank and the International Monetary Fund for debt relief; they require that a process of open debate be facilitated with civil society. Although the extent of measurement and the quality of participation in PRSPs are often questioned, they remain arguably the largest-scale development or health efforts to recognise the importance of community ownership and participation in policy-making, and the debates around them provide important input to participatory health programmes.21 ,22

Apart from PRSPs, analysis of participatory processes in small-scale project settings has primarily informed operational efforts to address marginalised communities.23 Such analysis is often qualitative and emphasises the importance of understanding and addressing power differentials which restrict the agency of the most marginalised,24 and of strengthening the capacity of CBGs representing marginalised people to undertake meaningful action.1 ,25 However, the question of how to understand and address scaled-up processes remains.26

We report here a methodology for measuring community mobilisation developed and used in a 10-year, large-scale HIV prevention intervention: Avahan, the India AIDS Initiative. The methodology was developed 5 years into the programme and was intended as a participatory27 process that would inform the programme—including the communities themselves—about progress made in community organisational development. Community in the Avahan programme refers to the high-risk individuals (female sex workers (FSWs), high-risk men who have sex with men (HR-MSM), transgenders (TGs) and injecting drug users (IDUs)) who come together through the programme at various levels: informal gatherings of no predetermined frequency at hotspots or drop-in centres; localised but formal meetings of high-risk individuals in CBGs to discuss the programme or community initiatives; and formal community-based organisations (CBOs) that meet monthly at the district level, levy membership fees and elect their own governing officers from the membership. The motivation for attending CBG meetings or being a CBO member varies from individual to individual, as it does in social movements globally.

In its first phase (2003–2009), partnering with implementing non-governmental organisations (NGOs) in six Indian states with high HIV prevalence, Avahan provided HIV prevention interventions to over 320 000 members of high-risk groups and about five million members of bridge populations (primarily clients of sex workers and long-distance truckers).28 ,29 The intervention package consisted of behaviour change communication, provision of STI services and prevention commodities (condoms, and needles and syringes where applicable), links to HIV care services and community mobilisation, and local, state-level and national advocacy.30 A description of community mobilisation efforts and results in Avahan is given in accompanying papers in this supplement.31–36

The Avahan community mobilisation logic model posits that participation and change by high-risk communities lead to stronger HIV prevention outcomes and the long-term goal of a sustained HIV response through CBGs.31 One major objective of Avahan's second phase (2009–2013) was transition to a model in which the government—and, to a limited extent, CBGs—would manage targeted interventions37 and CBGs would transform their own role by taking action to address their vulnerabilities and building strong organisations that would be sustainable beyond the lifetime of the Avahan programme.32 These broad outcomes operationally define community mobilisation and the type of capacity development that took place in Avahan.

Community mobilisation efforts are currently focused on strengthening members of high-risk groups to manage, own and sustain programmes that had begun as NGO-led interventions. To this end, Avahan's implementing NGOs have helped support the formation of CBGs. At the local level, these CBGs tend to be informally organised and play the role of a pressure group to reinforce and maintain quality of services; they may have democratic structures and form committees to provide feedback on aspects of the intervention. Over time, groups also form at a district level. Some of these district-level groups are legally registered CBOs. They have representation from the local groups and engage in structural interventions to influence public policy and claim the rights and entitlements due to community members as citizens.38 ,39

A Community Ownership and Preparedness Index (COPI) was designed by Praxis to establish a baseline for phase II of the Initiative and to provide ongoing monitoring. The detailed objectives were: (1) to analyse the implementation and effectiveness of community mobilisation in preparing CBGs for the transition of programme management and funding from Avahan to federal and state government and, in some cases, to CBGs themselves (‘transition readiness’); (2) to assist CBGs, Avahan and its NGO partners in reflecting on, planning and improving community mobilisation; (3) to learn lessons about how community mobilisation might be replicated in other large-scale contexts; and (4) to provide data for making inferences on community mobilisation, including its association with improved HIV prevention outcomes, when combined with other sets of data collected through management information systems and other surveys.

Methods

Methodological considerations

Several significant considerations underlying the COPI design are noted here and discussed further in the following section. The study was designed primarily to inform the programme, providing the same degree of qualitative insight that analysis of participation in small-scale settings has done. However, given the scale of the programme and the considerable variations in community mobilisation across it, a new approach was required, combining qualitative and quantitative methods to capture data on complex and sometimes abstract processes of community mobilisation. This was achieved in part by breaking down the community mobilisation process into components (referred to in this account as parameters and indicators) relatable to concepts from participation discourse. These parameters and indicators could be measured and aggregated at the level of individual CBGs as well as state and national levels. Aggregation was facilitated through the use of a weighting mechanism, where weights represented an estimation of the relative importance of various aspects of CBG performance. In addition, a method was devised to quantify qualitative data, such as the level of CBG members' participation in activities, or the extent to which activities contributed to structural change. This standardisation enabled the progress of individual CBGs to be measured from year to year, providing insights on community mobilisation within and across CBGs while capturing a level of nuance that could prompt meaningful reflection and course corrections at the local level.

The COPI was designed as a repeating cross-sectional survey, conducted annually across 32 districts in six states, using an index-based assessment of CBGs' capacity to manage programmes. The initial assessment established a baseline, to be followed by four rounds of surveys from 2010 to 2013.

Two principles drawn from participatory development influenced the process by which the COPI was designed. First, members of the high-risk communities participated actively in all stages of the design, including helping to determine the relative weight to be given to the various dimensions of community mobilisation. Second, a process for sharing data with the CBGs was built into the survey design. The process of data collection and analysis across six states was rapid (taking <3 months in total), and data were initially shared with each CBG as soon as they had been gathered, and then later through linked activities that facilitated participatory learning and planning. The intention was that CBGs could respond to the data and make decisions about the future course of their organisations and activities, and that the experience of participating in the study should be empowering and serve the CBGs directly.

Conceptual roots and the study design process

The development of the COPI was accomplished through a participatory iterative process, in which facilitated discussions and focus groups with high-risk communities were key to each stage. The process included: a review of background material and theory as well as learning from the experiences of Indian CBGs working in HIV prevention; design of the study framework and related indicators and parameters; weighting of indicators; and development and pilot testing of the survey tools. At each of these stages, the participation of high-risk community members was supplemented by input from various academic disciplines (eg, statistician, sociologist, anthropologist, demographer and gender expert). This required careful planning and took more time than if the design had been left to a small team of experts. However, it reflected the desire to develop a survey that reflected the community's own understanding of what was important to effective mobilisation.

The ‘powercube’ model, developed in part by John Gaventa, which focuses on power relations among different stakeholders to assess the strength of communities, was used initially as the basis for a discussion with CBG leaders in order to understand power relations for CBGs within the Avahan Initiative.40 ,41 Based on these discussions, a core design team, including high-risk community members and experts, identified three broad categories of stakeholders whom CBGs need to engage to address issues of vulnerability: (1) the project (HIV prevention intervention), including the implementing NGOs and programme components such as clinics and drop-in centres; (2) the state, that is, governmental bodies and authorities; and (3) wider society, for example, neighbourhood residents, religious leaders, politicians and gatekeepers (gatekeepers might include owners of sex establishments, clients of sex workers, drug dealers, intimate partners and law enforcement authorities). Table 1 uses the stages of the Avahan community mobilisation logic model to illustrate the expected trajectory of CBGs in relation to these stakeholders.31 Over the 10-year period of the Avahan Initiative, CBGs were expected to move towards the more advanced outcomes of Stage 3, where they would take ownership of elements of the intervention, develop procedures and spaces to claim their entitlements from the state, and assert their identity and rights as citizens within wider society.

Table 1

Expected evolution of the CBG's relationship to the three categories of stakeholders

Further discussion with CBG members, together with a review of the literature on assessment of NGOs42 and self-help groups43 and consideration of the transformative potential of participatory processes,1 identified characteristics of a strong CBG. These included leadership and the ability to strategise independently about all issues affecting the high-risk community; addressing the immediate and long-term needs of the community; transparency of management and decision-making; communication skills; going beyond HIV prevention and care; the ability of the CBG to remain connected to the wider high-risk community and acceptable to it; and scope for community members to participate voluntarily in the CBG's activities, especially those relating to realising their rights and entitlements. Working with this information, the design team collaborated with CBG members and experts to develop the evaluation framework illustrated in box 1.

Box 1

Community Ownership and Preparedness Index: framework of parameters and indicators

  • Dimension 1: Capacity in terms of strong leadership, governance mechanisms and decision-making systems

  • Parameter 1. Leadership that can address immediate and strategic needs of community members and has emerged out of the shadow of the implementing partner

  • Parameter definition: The capacity of community-based group (CBG) leadership to play a significant role in building solidarity among community members by being in the forefront in planning and implementing interventions to address crises, and by mobilising a critical mass of members to assert their identity and engage with issues through public action.

Indicators/subindicators

  • 1. Leadership team (LT) has demonstrated capacity to show solidarity during crises faced by community members.

  • 2. LT has demonstrated strength in mobilising community members to assert their identity and to engage issues through collective action.

  • 3. LT is capable of setting its own agenda and of emerging from the shadow of the implementing partner.

    • i. LT exists as an entity and meets regularly.

    • ii. LT independently sets agenda for its meetings.

    • iii. LT engages with the implementing partner over disagreements on a strong footing.

  • 4. LT has internalised the need for collective action for asserting the identity of the community members and realising their rights.

  • 5. LT has made efforts to develop second-line leadership.

  • Parameter 2. Governance system that is inclusive and participatory, with a democratic selection process and robust accountability arrangements

  • Parameter definition: The system of governance within the CBG is evaluated in terms of the relationship between leadership and the community members, including adherence to the principles of participation and inclusion.

Indicators/subindicators

  • 6. Participatory selection process for the leadership.

    • i. Participatory selection process for LT and office bearers.

    • ii. Participatory selection process for committee members.

  • 7. System in place for leadership's accountability to community members.

    • i. Leadership's accountability towards community members.

    • ii. Committees' accountability to community members.

  • 8. Inclusion of all groups in LT.

  • Parameter 3. Decision-making system that respects participation as well as the need for delegation and time-bound decision-making

  • Parameter definition: A defined system of decision-making for all processes; the system addresses the need for community ownership of decisions, is flexible, and addresses the need for delegation and time-bound decision-making.

Indicators/subindicators

  • 9. Defined system for decision-making, with CBG becoming the decision-maker.

  • 10. System to promote community involvement in strategic decision-making.

  • 11. Committees formed for crisis response and advocacy; committees are meeting regularly.

  • Dimension 2: Capacity to sustain itself through resource mobilisation and networking

  • Parameter 4. Resource mobilisation: the group is self-sustaining and has the capability to raise and manage resources, and individual community members are economically empowered

  • Parameter definition: In order to be sustainable as well as to have an independent stake in setting the agenda, the CBG must be able to mobilise resources from its own members as well as diverse other sources of funds, and to do so independently without requiring much support from the implementing partner.

Indicators/subindicators

  • 12. Strong, diversified resource base.

    • i. Financial.

    • ii.Non-financial.

  • 13. Entry into formal economy.

  • Parameter 5. Community collective networks: the CBG has the networking strength to assert its identity and to legitimise as well as realise its demands beyond its own members

  • Parameter definition: Community collective networks reflect the networking strength of the leadership team. This includes an increase in the outreach of the CBG in terms of increased membership as well as in forming loose networks with other CBGs and solidarity groups across districts and states.

Indicators/subindicators

  • 14. Regular increase in outreach.

  • 15. Networking with networks.

  • 16. Networking with other bodies.

  • Dimension 3: Capacity to lead and manage the targeted intervention programme

  • Parameter 6. Project and risk management

  • Parameter definition: Community leadership participates in various aspects of service delivery, not just as a beneficiary, but also by exercising greater control over decisions made in project spaces such as management committees. Community members are also able to create systems to detect and mitigate risks—legal and financial—in the management of the organisation itself, without requiring much support from the implementing partner.

Indicators/subindicators

  • 17. Leadership is aware of the requirements for managing organisations and can demonstrate its ability to do so.

    • i. Awareness of compliance with statutory requirements as well as systems to minimise legal and financial risks and risks due to adverse publicity.

    • ii. Demonstrated capacity to manage strong financial, accounting and administrative systems.

  • 18. Leadership is competent and confident in contributing towards project processes.

    • i. Awareness and implementation.

    • ii. Monitoring and strategising.

  • Dimension 4: Capacity to engage with state bodies and broader society to realise rights and address issues of stigma

  • Parameter 7. Demonstrable engagement with the state to secure civil and political rights and welfare entitlements

  • Parameter definition: Community leadership has been able to create and build new spaces of engagement with state bodies to realise the rights and welfare entitlements of the community members.

Indicators/subindicators

  • 19. LT has demonstrated capacity to deal with issues of violation of freedom.

  • 20. LT has demonstrated capacity to realise enabling rights.

  • 21. LT has demonstrated capacity to successfully realise entitlements for community members.

  • Parameter 8. Demonstrable engagement with other key influencers: the CBG can assert itself with respect to many other influential groups and individuals

  • Parameter definition: Community leadership is making itself visible, creating spaces in the social arena, and demonstrating and asserting its identity and rights.

Indicators/subindicators

  • 22. LT has demonstrated collective actions in engaging with gatekeepers to assert the identity of community members.

  • 23. LT has demonstrated collective actions in engaging with other organised groups and professionals and with opinion-makers to assert the identity of community members.

    • i. Engaging with other organised groups and professionals.

    • ii. Engaging with opinion-makers.

At the broadest level, the framework posits four dimensions considered essential to the transition readiness of CBGs: (1) leadership, governance and decision-making; (2) sustainability through resource mobilisation and networking; (3) project management; and (4) engagement with the state and wider society. These dimensions are elaborated into eight broad parameters, subdivided into 23 specific indicators (with between two and five indicators per parameter). Each indicator is a practical, operational expression of participatory concepts. Progress on each indicator is assessed through sets of questions. Each possible response to a question is precoded with a score; when the survey is conducted, the scores received for the questions relating to an indicator are totalled to arrive at an overall score for that indicator. Each indicator score is multiplied by the weight assigned to that indicator. The sum of the weighted indicator scores within each parameter yields a parameter score.

The scoring of questions was designed to assess multiple qualitative aspects of an activity. For example, instead of assigning the same score to each CBG where a particular activity takes place, a progressively higher score would be assigned if the planning for that activity was done by paid peer educators, by the CBG's leadership team working with peer educators or by the leadership team without peer educators. (The implicit rationale in this progression is that paid peer educators, although members of the high-risk community, are still part of the NGO-led programme and transition requires independence from the programme.) The same continuum applied to the execution of the activity. In some cases, the nature of the activity—informal, engaging a formal power structure or working for policy-level change—was also considered part of a progression in the levels of a structural intervention and was also scored on a continuum.

The procedure to validate the weights assigned to each of the 23 indicators is an example of the participatory approach to the design of the COPI. The numeric values expressing each indicator's relative significance to the overall process of community mobilisation in the CBG were debated in multiple forums, with experts including Gaventa and Chambers whose approaches have informed the programme, and with members of the high-risk communities, who negotiated together to determine the relative importance of indicators and parameters. In order to relate the parameter scores to transition readiness in a straightforward manner for discussion with the CBGs, a scale was used, dividing the maximum possible score of 100% into seven equally spaced bands (table 2). If a particular parameter score fell within one of the top two bands (‘Vibrant’), the CBG was considered to be ‘transition ready’ with respect to that parameter. (It was understood that the overall transition readiness of a CBG would depend on numerous contextual factors and involve negotiation with different actors, including the government body charged with taking financial responsibility for the intervention.)

Table 2

Community Ownership and Preparedness Index: seven-band scale for evaluating transition readiness of community-based groups on each parameter

Validation of the COPI

The predictive validity of the COPI was tested in several stages. It was not possible to test the validity before implementation, since the gold standard was not known. At that stage, only the content validity could be measured. First, a multiple-scenario analysis was carried out using the adopted design before the actual collection of the data, by inserting all possible types of potential answers into the cells. This analysis was helpful in checking the consistency and validity of possible responses to the questions, and whether the overall analysis aligned with the community mobilisation model for change.31 ,39 Second, a pilot test of the survey was conducted with four CBGs in the states of Andhra Pradesh, Maharashtra and Manipur, and when the data were scored using the COPI design, the results were found to be consistent with the current status of those CBGs as perceived by programme implementers. Third, the COPI tools were used to analyse the progress of one of the Avahan CBOs which had already demonstrated high achievements in most of the parameters. Further stages of validation are discussed later in this paper.

Implementation

Because of the limited resources available, it was not possible to conduct the assessment in all 83 districts where Avahan was working. A sample size of 32 districts (approximately 39% of the total) was fixed. The districts were selected using stratified random sampling and ensuring representation of: (1) all six states; (2) all seven state-level implementing partners; (3) all categories of high-risk group (FSWs, HR-MSM, TGs and IDUs) in each state where they were present; and (4) districts where Avahan was the sole HIV prevention intervention and districts where it was not.

In the majority of the districts selected, there was only one CBG or CBO operating at the district level. (There might be several smaller, informal CBGs at local levels, with formal representation in the district-level group.) Where there were two CBGs or CBOs at the district level, the one with the largest membership was selected for the assessment, and where there were three or more such CBGs or CBOs, the two with the largest memberships were selected. This method yielded a total of 36 CBGs and CBOs for assessment in the 32 districts.

Researchers were chosen based on several criteria: educational qualifications; past experience in field work; aptitude for field surveys and particularly for participatory approaches; tact and sensitivity in dealing with a stigmatised community; and commitment to the cause of HIV/AIDS prevention. The 25 selected researchers underwent intensive training to ensure that they understood how to implement the survey and the concepts behind the COPI design.

The assessment tool and methodology were discussed by Praxis's Internal Review Board, and a quality standards form was devised, incorporating questions related to the ethical dimensions of data collection. The researchers were required to complete this form each day that they were collecting data, and it served as both a self-monitoring system and a record for later appraisal. Praxis also formed a community-led ethics review committee representing FSWs, HR-MSM, TGs and IDUs. The committee reviewed the assessment tool and methods and served as an external body to ensure that the monitoring was conducted in a way that was ethically acceptable to the community.

The field work for the entire survey took place over a period of 3 months. The researchers, working in teams of two, conducted 5-day-long visits at each CBG. Questionnaires (some completed in written form ahead of time, but most completed through face-to-face discussion during the visits) were given to four groups of respondents, identified as the primary stakeholders mandated with preparing CBGs towards transition readiness: (1) the CBG's leadership team and office-holders; (2) members of CBG committees; (3) staff of the implementing partner (NGO); and (4) community members. The survey was scheduled so that respondents were not required to be present for more than 2 days to make it as easy as possible for them to participate.

Data collection at each CBG took place over three-and-a-half days. Typically, 1 day was spent with the leadership team (all members of the executive committee), half-a-day each with the office-holders (president, secretary and treasurer), NGO staff and committee members (crisis response and advocacy committees), and 1 day with two subgroups of site-level community members. The final one-and-a-half days were reserved for gathering any incomplete information and for a consultation with the respondents who were presented with a preliminary analysis of findings from the data collected. This feedback was intended to add value to the overall process by minimising sampling biases or errors and providing the CBG and implementing partner with immediate data to inform their plans and programmes. Contact with community leaders was maintained so that further data could be requested or submitted via post or email as needed. A report, translated into the local language, was later sent to community leaders for their review, and additional data use workshops were held.

Data collection proceeded simultaneously in all the sample districts, and data were entered directly in a computerised database developed by Praxis using Filemaker Pro (Santa Clara, California, USA) platforms, which allowed concurrent analysis of data for individual CBGs. This necessitated a thorough monitoring system at all levels of data collection and entry to ensure the completeness and quality of data and to promptly resolve any concerns. The researchers completed a checklist, consisting of an evidence sheet, probe questions and a tool sheet, to ensure that no requisite data were omitted.

State coordinators performed quality checks and cleaning of data at the end of each day of field work and informed the field researchers of any gaps. Data were then analysed by a team of state and national coordinators whose concerns were to ensure overall consistency in the gathering of data and other evidence.

Wherever possible, documentary evidence was collected to cross-check statements made by the respondents. This evidence included newspaper reports, audit reports, statutory returns filed, minutes of meetings and copies of statements made to the police or requests filed under the Right to information Act. Documentary evidence was reviewed first by the researchers and then by the state coordinators before data collection was considered complete.

Discussion

The COPI was designed to enable Avahan, its implementing partners and CBGs to systematically monitor the transition readiness of CBGs. The survey informed programme management at scale, contributed to a participatory management approach and provided objective information to CBG members for facilitating decisions about how to make the CBG sustainable, access resources and achieve other goals. Results are presented in an accompanying paper in this supplement.44

Avahan attempted to move beyond the shortcomings of previous efforts to measure participation in large-scale programmes by using the theoretical premises of participatory development to inform a survey framework which would provide insights into the complex processes being monitored. Concepts such as leadership quality, governance and decision-making, at once abstract and interdependent, were incorporated into measurable parameters with multiple indicators that were scored to measure relative transition readiness.

Similar to other social development index methodologies, for example, the Human Development Index,45 the COPI assigns weights to different indicators and parameters reflecting their relative importance to transition readiness. However, in contrast to the Human Development Index, a participatory process was integral to determining these weights. Consultations were held with experts on community mobilisation from the Institute for Development Studies in the UK, and with high-risk community members themselves. The final weights for each indicator were arrived at by averaging the values given by each of these groups. While there was inevitably an element of subjectivity to this process, it was mitigated by ensuring that the process was participatory and transparent, and by encouraging experts and community members to engage with and challenge one another's rationales for the assigned weights. Including the high-risk community and experts in the process also gave the COPI legitimacy in their eyes when the data were analysed with them.

One further aspect of the methodology attempted to counter the subjectivity involved in assigning weights to parameters and indicators. All primary data were preserved distinct from the scores and weights assigned to them for analysis so that if there should be a need to revise weights at any point because of changes in the external environment or in conceptual understandings of community mobilisation, it will always be possible to retrospectively apply revised weights and obtain the appropriate new scores.

With 3 years of data now available, further exercises are underway to test the predictive validity of the COPI. The tools are being tested with CBGs within and outside the HIV sector whose profile and strengths are known. With respect to the validity of the assigned weights to indicators and parameters, a statistical analysis was undertaken in which an independent set of statistically derived weights was evolved to compare with the weights used in the COPI, which were developed through discussion with stakeholders and experts. It was found that the COPI weights turned out to be very close to the derived weights. The statistical analysis will be repeated using 2011 data. There is therefore now the option of using two kinds of weights: those based on discussion and negotiation with stakeholders and those that are statistically derived.

Among the considerations in the study design were dynamics within the CBG as well as between the CBG and the non-community implementing partners (the local and state-level implementing NGOs). The standard practice in evaluation is that data should be collected objectively and in a way that precludes bias on the part of the data collectors and those who are providing information. This level of objectivity is similarly needed in capacity development. The literature points to a power imbalance between NGOs and the communities they work with to develop capacity, and suggests that an organisation doing capacity development work with communities should not also collect data on that process since it would have little incentive to reveal any weaknesses, and communities might therefore not accept the legitimacy of the data.25 ,46 In the case of the COPI, data were collected by Praxis, an organisation that was not involved in the capacity development of the CBGs being surveyed; it was thus expected that power dynamics between the NGO and community could be spoken about more openly.

A further innovation in the COPI methodology is that it was intended to make the process of monitoring part of the community mobilisation programme itself. The survey was designed to empower CBG leaders: its content informed them about programme quality, rights and entitlements, and approaches to addressing stigma, and the survey process itself, using intensive interviews with CBG leaders and members, created discussion on these issues. When the survey data were presented to the CBGs, the facilitated discussions quickly moved from considering the abstract scores to reflecting on their operational implications. These discussions were designed to challenge power dynamics, expand the vision of CBGs to opportunities beyond the programme and build collective agency. The experience of implementing the survey validated the design's effectiveness as a participatory action tool and demonstrated that monitoring can in effect be a useful intervention in itself.

What is already known on this subject

  • Community mobilisation can improve outcomes of health interventions, including reducing vulnerability to HIV among at-risk populations.

  • Attempts to foster effective community participation in large-scale health interventions have met with mixed results.

  • Attempts to clearly define and measure community mobilisation in large-scale programme settings have largely been considered inadequate.

  • Attempts to measure community participation through quantitative approaches have largely been restricted to behavioural surveys, and complementary efforts to undertake quantitative monitoring have been problematic.

What this study adds

  • It is possible to meaningfully involve high-risk communities in the design and implementation of an assessment of their own levels of mobilisation in a large-scale programme.

  • It is possible to design a standardised, weighted index to assess the complex components of community mobilisation, incorporating the community's own perspective on the relative importance of these components.

  • A quantitative cross-sectional monitoring approach can produce valuable results in the measurement of community participation when it seeks means of verification, employs multiple sets of indicators to examine complex issues and triangulates data to correct for social desirability bias.

Acknowledgments

Tom Thomas, Pradeep Narayanan, Joseph MJ, Anindo Banerjee, Nalini Jamela (FSW) and Priya Babu (TG) formed the Praxis core design team. The team was assisted by Tisha Wheeler, Usha Kiran, James Moore and Gina Dallabetta (Bill & Melinda Gates Foundation), Ramanathan TV (statistician, University of Pune), Rick Davies and Geeta K (expert on gender, sexuality and violence). The tools were designed after consultation among community groups in Kolhapur (Maharashtra), Chittoor (Andhra Pradesh), Ramanagaram (Karnataka), Nellore (Andhra Pradesh) and IDU groups in Imphal (Manipur). A detailed discussion on quality of tools and results was conducted with the community leadership team of Ashodaya (Mysore, Karnataka). A meeting of community leaders from CBOs in the four states was organised to assign weights for different indicators and parameters. We are grateful for the time and intellectual guidance that these community members offered in the design of the tool. Peer review consultations were held with Robert Chambers, John Gaventa, Kim Blankenship, Christine Galavotti, Niranjan Saggurti. James Baer (independent consultant) and Sowmyaa Bharadwaj (Praxis) assisted in the drafting of this paper.

References

View Abstract

Footnotes

  • Linked articles 200562, 200487, 200465, 200475, 200511, 200832, 201065, 200478, 201081, 200508, 200514.

  • Funding This study was funded by the Bill & Melinda Gates Foundation through the Community Mobilization Monitoring Project. The views expressed herein are those of the authors and do not necessarily reflect the official policy or position of the Bill & Melinda Gates Foundation.

  • Competing interests None.

  • Provenance and peer review Commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles