While the “scientific community” holds to the principle that all public health must be evidence based, in practice the effectiveness of many health promotion interventions and programmes is not properly assessed.¹ Among the reasons for this situation, the lack of an adequate control group is commonly an important obstacle for a careful evaluation of effects. An equivalent control group is rarely available to assess the impact of large scale health promotion campaigns or new policies, which are often dependent on time trends comparisons. While time trends sometimes show impressive changes, often historic trends or other factors influencing outcome are not properly taken into account, resulting in a very simplistic appraisal of the programme’s utility. As a consequence, the interpretation of changes seen on timelines becomes a rather subjective appreciation.

Another main barrier to impact evaluation is the complexity of many health promotion interventions, especially those targeting human behaviour change. Unlike therapeutic and diagnostic procedures, preventive interventions that aim to promote healthy lifestyles focus on knowledge, attitudes, and beliefs, as the “predisposing factors” of a certain behaviour.² These factors are deeply rooted in the social and cultural context; that is, programmes focusing on promoting condom use among adolescents may be seen and implemented differently in different countries and different cultures. In addition, these programmes must address other changes in personal skills and environmental changes that facilitate and reinforce the healthy behaviour. As a result, these interventions have many components, some of them closely tied to specific environments, and therefore they are difficult to standardise. Moreover, when an effect is observed, it might be difficult to attribute specific contributions of the programme’s different components.

Overall, we must admit that very often evidence is either not available or not possible to obtain, at least under the accepted paradigm of evidence. Therefore, we need to find a new frame for the evaluation of complex health promotion interventions. In the past few years, many attempts have been made to clarify contributions of non-randomised interventions. In 2004 a guideline of how to improve and standardise the report of results from non-randomised trials was published. The Transparent Reporting of Evaluations with Non-Randomized trials (TREND) suggests a list of items that should always be reported to facilitate understanding the theories and methods, including design and potential biases, and therefore better clarify overall assessment and evaluation.^3,4 While this statement is just a first step, it may well be useful to improve the contribution and acceptance of evidence coming from designs different from randomised controlled trials (RCT) in the public health field. In this regard, plausibility and adequacy statements have been invoked to assist judgement of health promotion activites when randomised clinical trials are not possible.⁵

In practice we will often face the situation of planning an intervention that we will not be able to evaluate properly, and it might be unclear if previous evaluations may be taken for granted in our specific context; this question, related to external validity—how close the previous evaluation is in time, population, and methods—is not easy to answer, as, by nature, complex interventions will rarely be repeated in exactly the same way. If we consider that previous evaluation cannot be taken as a valid reference, then we must assess what kind of evaluation is possible. When possible, we will use a strong design that can resist threats to internal validity, using an equivalent control group. However, when these designs are not possible, we must decide on what ground will we go ahead with the intervention; can we assume a “principle of prevention”, so that under certain circumstances, we can expect a positive effect even if it is very unlikely that we can absolutely verify it? I believe we can and we should, especially when the alternative is to ignore the problems and limitations of the current situation. In any case, it might be better to make explicit the rules and the reasons for the decision taken. For example, we can state that some kind of (positive) effect has been proved by at least one of the components of the programme, or that an explicit theoretical model of effect must be provided in advance, which must be checked by any combination of process and impact, quantitative and qualitative evaluation, to provide the best possible ground for utility. Also, when previous evaluations are to be taken as reference, an explicit justification to expect that changes and adaptations from the original programme (or any of its components) as it was originally evaluated will not reduce effectiveness may be requested.

No doubt the process of standardising and developing appropriate programme evaluation is going to be a long and complex endeavour, but it will eventually prove to be worthwhile. If not the ideal, we should do our best to provide the best possible evaluation for every health promotion programme and intervention. We can no longer pretend RCT are “the way”, ignoring the fact that most public health interventions and programmes do not fit into their rules. There are too many efforts and resources at stake. We cannot maintain the gap between “believers” and “sceptics” regarding health education and health promotion: both groups would benefit from good evaluation, whenever possible, to show that health promotion is not useless nor the “panacea”. There is room for compromise, just as there is with everything in life.