OBJECTIVE To analyse dilemmas and challenges in health promotion research, and to generate ideas for future development.
METHOD The analysis is based on authors' experiences in working in the field of research and action in health promotion and on experiences of others as found in literature.
RESULTS The assumptions underlying scientific research as based in the biomedical design are difficult to meet in community-based health promotion research. Dilemmas are identified in relation to the possibility of defining the independent and dependent variables beforehand and the intermingling of these variables (the intervention and outcome dilemma), the difficulty in quantifying the desired outcomes (the number dilemma), and the problem of diffusion of the programme to the control group (the control group dilemma).
CONCLUSION Research in health promotion has specific reasons to reconsider the approach towards research, the selection of outcome variables, and research techniques. Strategies and methods to make activities and their outcomes clear are discussed and criteria to judge confidence and applicability of research findings are presented.
- health promotion research
- research dilemmas
- research challenges
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
The 20th century has seen greater health gains for populations of the world than in any other period in history. The greatest advances in health have been made through a combination of structural change and the actions of people.1 The health promotion movement2-4 advocates this broader focus. Compared with the medical paradigm this movement has wide ranging consequences for working methods, whereas the consequences for research are probably even more complicated.
Research in any field is oriented to the production of knowledge to support the development of that specific field. Similarly, research in health promotion should be oriented to the improvement of health promotion practice. There is a need for better understanding of health risk factors and salutogenesis,5 but also of the role of the environment in creating or damaging health.6 In addition we need “know how” to be able to reorientate health services, to reduce inequalities in health, and to achieve successful community participation and intersectoral work. This includes information about the health impact of social and economic policies, but also about processes, for example, the process of intersectoral collaboration and community participation, the process of planning and implementation of activities, and the process of policy development. Research in health promotion means to investigate different issues, and using other methods or other combinations of methods than those traditionally used in biomedical research.
The objective of this article is to analyse problems and opportunities in health promotion research, and to generate ideas for future development. The analysis is based on our experience of working in the field of research and action in health promotion and on the experiences of others as found in literature. All of these focused on change at the physical and social environment, instead of on the mere focus on transformation of knowledge and information to people.7-12
Dilemmas in health promotion research
The biomedical research paradigm is based on the assumption that the independent (the intervention) and dependent variables (expected outcomes) are well defined beforehand, and that the dependent variables can be expressed in meaningful numbers, preferably measurable on at least interval scales. Expected outcomes are expressed in cause-effect relations (for example, intervention leads to change in specified determinants of behaviour). In addition, internal validity and external validity are considered important.13 14 These assumptions, however, seem to be difficult to meet in community-based health promotion. In fact, the difficulties emerge from the philosophy of working according to the principles of health promotion, causing some real dilemmas.
THE INTERVENTION AND OUTCOME DILEMMA
The first dilemma relates to the definition of the independent and dependent variables. In the biomedical approach the independent variable can be rather unambiguously described beforehand. An organisation develops an intervention aiming at a certain type of behaviour, using a self decided and well described strategy, implements it, and evaluates its effectiveness. In health promotion however, developing an intervention is much more complicated. Firstly, besides changing individual behaviour health promotion also aims at changing the conditions for change, related to social, cultural, and organisational factors. Secondly, to achieve both objectives, health promotion is a participatory process. Interventions are developed, implemented and evaluated together with stakeholders from different organisations, including citizens. The participants represent different disciplines, with different backgrounds, philosophies, values, and practices.15 They are often not used to working together and have to go through a process of social learning. Hence, intersectoral collaboration and community participation are embedded in the independent variable, but also have to be considered as outcome variables.
A further consequence of health promotion practice is that, even though in many programmes the topic is decided upon, the interventions cannot be described carefully at the onset but are decided upon along the process with stakeholders. According to Frankish and Green this increases the feelings of ownership16 and in turn, ownership increases capacity (competence) and promotes programme maintenance. As such change can be perceived as a social movement at non-governmental level, which, once the movement starts, accelerates.17 From a research perspective however, it constitutes a real dilemma. Deciding about activities, settings and strategies while the programme is running hampers the possibility to adequately select dependent variables for the study beforehand. Moreover, it is almost impossible to isolate the contribution of single elements of the interventions to any observed change in outcome variables. This problem occurs in many community-based health promotion programmes,18 and is a direct consequence of the approach. Theoretically, the solution would be to use a phased implementation in which first the effect of one activity in one setting is assessed before the second and subsequent activities begin. In practice however, it is no solution. Firstly, because there is a good chance that the second intervention interacts with the first one. Whether one type of intervention actually influences the effectiveness of the others remains therefore empirically an open question.19Secondly, because of the enthusiasm of participants, interventions take on a life of their own: once contrived, the participants want to conduct the activity as soon as possible. We have learned that such quick implementation is an essential factor in sustaining community involvement.10 11
THE NUMBER DILEMMA
In the biomedical approach, outcomes generally relate to changes in knowledge, attitudes, self efficacy, and behaviour. To measure such changes, instruments are developed and validated, in which the variables are expressed in numerical, preferably interval, scales, allowing sophisticated statistical analysis. In health promotion however, outcomes also relate to changes at the social, political and environmental level,20 that is changes in actions that support healthy lifestyles, the establishment of networks for intersectoral collaboration and community participation, and eventually, the empowerment of people and communities. However, and given the fact that most researchers are trained to produce quantitative data, changes in these spheres cannot easily be expressed in numbers, and sophisticated statistics are not applicable. The traditional research methodologies and techniques are insufficient to indicate health promotion success in those less countable areas.
THE CONTROL GROUP DILEMMA
Community-based health promotion programmes are carried out in field settings. Consequently, several external factors can interfere with the interventions, comprising a severe threat to the internal validity. To account for such threats, a randomised control trial would offer the best solution, but randomisation is usually impossible in those settings. Most appropriate then seems to be a quasi-experimental design, that is a pre-test/post-test design with an experimental and control group.13 But settings are generally open to the public at large, and people living in the control areas have access to the activities as well. Moreover, mass media like local newspapers, radio and television programmes may often report on the activities. Although awareness of the project and participation in it are relevant indicators for community change,21 from a research point of view the situation is rather inconvenient. It constitutes the dilemma of rigor and relevance,22 a struggle between the demands of scientific research and the knowledge of what constitutes good health promotion practice. However, if in the end the research-design reduces the effectiveness of the intervention, especially because it runs counter to the valued processes in health promotion, it is really questionable if this design is a suitable option.1 23
Challenges in health promotion research
We do not consider the dilemmas as constraints but wish to see them as challenges. We agree with Dean and Dawson24 that understanding the complex working of many forces needs to be the knowledge goal, rather than rigor and parsimony. Research has to produce knowledge about effectiveness of interventions in terms of individual behavioural change, but also about the importance of the principles of health promotion and about how and why health promotion programmes work. It is necessary to show that health promotion can be successful in order to communicate about the project, to continuously involve stakeholders and to obtain involvement and support of politicians and decision makers. Evaluation findings serve legitimisation and accountability, and are critical for the allocation of monetary, staff and other sources.25 Therefore, research in health promotion, regardless of the focus (for example, nutrition, healthy prisons, child development), has special reason to reconsider the approach towards research, the selection of outcome variables, and research techniques.
RECONSIDER THE ROLE OF RESEARCH
What is good for science and the individual scientist may not be good for the target group of research. What needs consideration is not what makes research good within the tenets of dominant theories (including research methods) but what makes it good for health promotion.26 One of the aims in health promotion research therefore, should be to look for a more suitable research paradigm. Traditional research models are based on the wish to discover, predict and control natural phenomena. It is a process of reductionism, which entails breaking down components of a complex world into discrete parts, analysing them, and making predictions based on interpretations of these parts.27 28 However, this breaking down opposes the holistic nature of health as we wish to see it in health promotion.29 Community-based programmes assume that there is a synergy among components, with the whole being as greater than the sum of its parts. In contrast with a one intervention-evaluation situation, it involves an ongoing process of decision making, requiring a flow of regular inputs. Methods in research have to be determined, among others, by the purpose of the study, the context and the setting, the theoretical perspectives (including “local” theory), the applicability of the measurement tools, and the input of community participants.30
In health promotion it is assumed that the positive effects particularly depend on intersectoral collaboration and public participation. These principles are essential for health promotion activities, but also for health promotion research. As the community is the focus of research, there should be dialogue on an equitable basis between researcher and community.31 This point is very well expressed in the eight key principles of community-based research as formulated by Isreal et al.30 These are: it recognises the community as a unit of identity, it builds on strengths and resources within the community, it facilitates collaborative partnerships in all phases of research, it integrates knowledge and action for mutual benefit of all partners, it promotes a co-learning and empowering process, it involves a cyclical and iterative process, it considers health from both positive and ecological perspectives, and it disseminates findings and knowledge gained to all partners.
A suitable research paradigm therefore would be based in participatory action research. This includes involvement of community members in research activities, and it enables an ongoing analysis of community strengths, resources, structure and dynamics. Participatory research attempts to negotiate a balance between developing valid generalisable knowledge and benefiting the community that is being researched.32 Incorporating the knowledge and expertise of community members enables to improve research protocols, and thereby it increases the opportunity to create scientifically validated knowledge.26
Participatory action research serves two functions: (a) a research function, to show processes, progress and results, and (b) an action function, that is a tool for action, reflection, discussion and decision making. Essential is that the results of each research step are immediately fed back into the project. By its very nature, participatory action research involves an active role of the researcher in designing and implementing the intervention and one has to accept that the researcher is not the external observer as in the traditional scientific research models. In contrast, as Whyteet al 33 state, the researcher is constantly challenged by events and ideas, information and arguments posed by the project participants. Participatory action research creates active support for the results of the process of inquiry and therefore, greater commitment to change as well as the greater likelihood that ideas will be diffused.34
Research in health promotion means the investigation of different issues, and using other methods than those traditionally used in biomedical research.
The traditional research methodologies and techniques are insufficient to indicate health promotion success in less countable areas.
Research has to produce knowledge about effectiveness of interventions, but also about how and why health promotion programmes work.
The mere fact that intersectoral action takes place can be considered as a success factor, as well as the intention to continue collaboration.
Single qualitative research techniques may give limited insight, but combinations give a rich picture of processes, achievements and conditions for improvement.
Health promotion research functions as a tool to measure change and innovation, but also to facilitate these outcomes.
RECONSIDER THE DEFINITION OF OUTCOME VARIABLES
Objectives that relate to positive changes in knowledge, attitudes, behaviour, and measures of capacity building,35can only be achieved in the long run36 by requiring regular and multiple input of information and action. If the achieving of these objectives would be the only criterion to show that health promotion initiatives work, then difficulties arise. In general, health related behaviour is complex, and change can only be achieved if certain conditions in the environment change as well. Moreover individual level measurements are too expensive.19 23Another important outcome of health promotion is sustainability,37 38 which means that networks and activities become a permanent part of the local community structure. In health promotion it is necessary to measure not only final or distal outcomes in health, equity, empowerment, etcetera, but also those intermediate or proximal ones.19 39 40 Community level indicators can also be based on observations of aspects of the community other than those associated with individuals. For example, policy and regulation indicators can include laws on tobacco use, policies on physical education or guidelines for menu and food preparation and these indicators provide a good starting point for outcomes at community level.41 In addition, the mere fact that intersectoral action takes place can be considered as a success factor, as well as the intention of people and organisations to continue collaboration.
RECONSIDER RESEARCH METHODS AND MEASUREMENT INSTRUMENTS
In health promotion, health literacy (knowledge, attitudes), social mobilisation (intersectoral collaboration, networking, empowerment, community change), environmental change and public policy and organisational practice (policy statements, legislation and regulations, institutionalisation of health promotion programmes) are considered to be preconditions for outcomes at the individual level.1 These processes and their outcomes cannot easily be measured in the usual quantitative way, and therefore require both a quantitative and qualitative orientation.42 43 In our experience, besides quantitative instruments such as validated questionnaires to measure determinants of individual behaviour, we applied several qualitative instruments, such as open interviews with key informants, and focus group interviews44 with professionals, volunteers, and citizens. In addition, we applied rapid appraisal techniques45 and stakeholder analysis techniques, such as the stakeholder interest matrix and the stakeholder importance matrix.46 These instruments provided information about sectors related to the health problem, networking experience, possible contribution of the stakeholders, their possible role and function, mutual expectations and conditions for participation. The processes of collaboration and participation were on the one hand assessed with the help of quantitative indicators, such as the numbers of collaboration structures, stakeholders attending meetings, and contacts between the groups; and the amount of time spend on the programme. The quality of collaboration on the other hand was assessed with the ladder of participation27 (the extent to which stakeholders participate in action and decision making), and the participation measurement instrument47 (functioning of the network in terms of leadership, organisation, resource mobilisation and management). These measurements served to evaluate the state of the art of the network and to visualise changes in participation in the course of time. In addition, discussing the results in workshops (feedback) with stakeholders served an action function, facilitating reflection about the processes of participation and collaboration, enabling participants to identify elements for improvement in future collaboration. Each of the mentioned techniques may give limited insight, but the combination gives a rich picture of processes, achievements and conditions for improvement.
RECONSIDER CRITERIA TO JUDGE VALIDITY
Conventional empirical research uses four criteria in judging confidence and applicability of the findings: internal validity, external validity, reliability and objectivity; criteria based on the ground assumptions of the traditional research paradigm.13 14 Participatory research encounters many difficulties in meeting these norms.30 Even if the practical value of the results may be clear, the question is whether the findings of such an inquiry are of good quality. In our experience this has been a point of debate over and over again. Based on these discussions, criteria to judge confidence and applicability of the results have been developed.26 The criteria are based on the work of Caplan and Nelson,48 Guba,49Chandler,28 Engel and Salomon,45Pretty,50 and, maybe more important, on the underlying principles of health promotion.51 52 The criteria pursue the route of planning, implementation, adjustment and reconsideration of health promotion activities and follow the line from practice to theory. In addition, they account for several of the above mentioned arguments.
(1) Look at the information from different perspectives, methods and sources
Triangulation: A promising approach to improve confidence in research findings is related to the concept of “triangulation”. Triangulation simply means using more than one approach to answer the same question. It refers to a cross check of information, using multiple sources, multiple methods and multiple investigators.53 54 Multiple sources(data source triangulation) refers to the use of multiple copies of one source such as interviews with members of one social group, or to the use of different sources about one topic, for example, interviews with consumers, health and welfare professionals working in a community, and volunteers about perceived nutrition related problems.Multiple methods (methods triangulation) refers to the comparison of results derived from a range of methods. For example, comparing the results of observation in schools with those of interviews with parents and group discussions with teachers. Once a proposition is confirmed by one or more method, the uncertainty of its interpretation is greatly reduced. The idea ofmultiple investigators (researcher triangulation) is especially strong in participatory research: those involved in activities are involved in the inquiry as well. It includes, as mentioned before, a variety of professions, backgrounds and standards, which increases the range of perspectives and biases. When the participants agree on interpretations of results, the threat of biased interpretations is much less. Combining information from different quantitative and qualitative sources to assess consistency in results can provide powerful evidence of success, as well as providing insight in the process of change in populations and organisations.1
(2) Check information continuously with participants at the local level (different sectors)
Continuous collaboration and debate: The variety of actors involved in community-based health promotion each have their own specific domain of knowledge and information, philosophy, objectives, standards, need for domain protection, and their own finite horizons.15 Exchange of ideas and debate about differences leads to sophisticated information and definitions of the situation at hand and gives insight in situations asking for improvement. This can result in a jointly agreed upon research and an action agenda in which the participants recognise their own position and in which they feel involved.
Participant checking: This involves the feedback of interpretations and conclusions drawn by researchers to the people with whom the original information was constructed. The objective is to test the data, interpretations and conclusions. If the reconstructions by the researchers are recognised by the participants as adequate representations of their own realities, the credibility of the findings is established. Participant checks can occur during interactive analyses and inquiry, but also during formal presentation meetings towards the end of the research. Participants have the opportunity to hear a summary of what investigators have learned and constructed, to investigate discrepancies, to challenge findings, and to volunteer additional information.
Differing views about the interpretation of results: Participatory research involves a range of actors in gathering information, analyses and interpretation. Discussion of the findings and their meaning from different perspectives can result in joint interpretations, but also lead to disagreement. Interpretations with a high level of agreement can be considered reliable, whereas disagreement should lead to further inquiry.
(3) Check information at the (inter)national level (different cultures)
The forum: Certainty about the value and validity can be further increased by external presentations, enabling the critical exploration of aspects of the inquiry that might still be implicit in the minds of the individual researchers and therefore subject to bias. In such instances, collaborative projects give opportunities to discuss them extensively.
Multiple cases: By multiple cases we mean parallel investigations in different settings. They are essential as they can demonstrate replication. Experience is affirmed if other research teams proceed with parallel investigations using similar techniques and come up with similar results. The Super-project is an example of a multiple case study.10 55 Conducting comparable studies in different situations has the potential to draw conclusions about the quality of achievements and the processes in force in community-based projects, but also about the utility of (new) research techniques. This offers prospects to develop strategies, which are useful in other communities. Multiple case studies provide a basis for generalising the results to other situations, and that is exactly the meaning of the criterion of external validity.
The steps are essential parts of the working process of health promotion. They motivate participants for action, which includes capacity building. The practical relevance therefore is clear and the steps are a basis for scientific validation as well. They provide the criteria against which the information can be judged and much like statistical analyses, provide grounds for judgement in conventional methods.
Research plays an important part in stimulating and guiding action, in providing feedback, “food for thought”, and in enabling reflection and discussion. As such, research not only functions as a tool to measure change and innovation, but also to facilitate these outcomes. The routes through which positive outcomes are to be achieved are complex, related to individual factors, but also to a great extent to social, cultural, organisational and economic factors. Research therefore has to be methodologically eclectic, selecting methods that are most likely to illuminate issues. Such an approach to research makes relevant contributions to both practice and science and thereby to the necessary further development of a comprehensive theoretical basis for health promotion processes.12 56
Conflicts of interest: none.