Aims: To evaluate the effectiveness of an evidence based group educational outreach visit on prescription patterns of non-steroidal anti-inflammatory drugs (NSAIDs) in primary care.
Design: Randomised controlled simple blind trial, with randomisation into three groups: experimental (evidence based educational outreach visit), placebo (conventional education session), and control (without intervention).
Setting: The 24 primary care centres of the National Institute of Healthcare Network in a rural province of Aragon, Spain.
Participants: The 24 primary health care teams of the network, with 158 general practitioners (GPs). The teams were randomised into the groups, experimental (8 teams, 48 GPs), placebo (8 teams, 54 GPs), and control (8 teams, 56 GPs).
Intervention: Experimental group: one group educational outreach visit, conveying data based on a systematic review of the literature that was reinforced with printed material; placebo group: one non-structured educational session; control group: no intervention. Both educational sessions emphasised that there are no differences in the effectiveness of the NSAIDs reviewed (diclofenac, piroxicam, and tenoxicam); a recommendation was made to prescribe diclofenac over tenoxicam because of price differences.
Main outcome measures: Changes in the number of packages prescribed for each of the drugs and changes in the cost per package of NSAIDs prescribed during the six months before, and after the intervention.
Results: There were no differences in the basal characteristics of the three groups, except for the number of prescriptions during the six months before the intervention. Prescriptions for NSAIDs decreased homogeneously in the three groups. For tenoxicam, the experimental group reduced prescriptions by 22.5% (95%CI: 34.42 to −10.76), compared with a reduction of 9.78% (95%CI: −17.70 to −1.86) in the placebo group and an increase of 14.44% (95%CI: 5.22 to 23.66) in the control group. The average cost per prescription decreased by 1.91% (95%CI: −0.33% to −3.49%) in the experimental group, 0.16% (95%CI: −0.27% to −2.93%) in the placebo group, and rose by 1.76% (95%CI: 0.35% to 3.17%) in the control group.
Conclusions: Evidence based educational outreach visits are more effective than no intervention at all. Results suggest that evidence based educational outreach visits are incrementally more effective than conventional educational sessions, which in turn are more effective than no intervention at all.
- educational outreach visit
- drug prescribing
- intervention trial
- primary care
Statistics from Altmetric.com
During the past few years different strategies to modify or improve medical practices have been implemented and evaluated.1–4 These have included distributing educational materials and guides of practice, holding conferences, informational sessions, educational outreach visits, interventions by local opinion leaders, meetings to hear patients' opinions, conducting audits with or without feedback, sending reminders, and other activities borrowed from the field of marketing, fostering processes to reach consensus on the local level and various combinations of all of these interventions. Interpreting the results of these interventions is a complex task, making it difficult to draw conclusions concerning their effectiveness, the scope of their expected effect or their comparative advantages.3–8
An educational outreach visit (academic detailing) consists of a visit to a healthcare provider in his or her own setting by a trained educator.9 A recent review has shown that these visits are effective in modifying the behaviour of healthcare professionals, and are particularly useful in changing prescribing practices.9 However, little has been studied of the effectiveness of this technique applied to groups of professionals,10–12 or when recommendations are made based on the systematic review of scientific evidence.13 This last aspect is relevant, because the additional effort of conducting a systematic review of the evidence to prepare the visit should translate into an increase in the effectiveness of these visits compared with conventional educational sessions.
The object of this study is to evaluate the effectiveness of group educational outreach visits that convey information based on the systematic review of evidence in modifying prescription patterns for non-steroid anti-inflammatory drugs (NSAIDs) in primary care.
Pragmatic, simple blind trial, with random assignation to one of three groups: experimental group (participating in an evidence based educational outreach visit), placebo group (participating in a conventional educational session), and control group (with no intervention).
The study was conducted in the primary health care network of the Instituto National de la Salud (National Institute of Healthcare) in Teruel, a rural province in Aragon, Spain. The network is comprised of 26 primary care teams (24 at the start of the study), which usually include six or seven general practitioners (GPs), one to three paediatricians, and 6 to 10 nurses. Among the relevant features of this health care system administered by the Spanish government are the universal nature of coverage, the cost free status of care (except 40% copayment of pharmaceutical products for non-retired users), funding of health centres by means of global budgets, and the semi-civil servant status of the doctors, who are paid by salary.
The 24 primary care teams that existed in Teruel at the start of the project participated in the study. These included a total of 158 GPs. By using a table with randomised numbers, the teams were assigned to one of three groups: the experimental group (8 teams, 48 GPs), the placebo group (8 teams, 54 GPs), and the control group (8 teams, 56 GPs). All of the groups were blind to the randomisation.
Systematic review of the effectiveness of NSAIDs
Three active agents from the group of NSAIDs were selected for review: piroxicam, tenoxicam, and diclofenac. These drugs were selected for the following reasons: (a) they are frequently used in primary care; (b) they are drugs that are prescribed and managed by GPs, with very little input from specialists; (c) they are practically identical in terms of effectiveness, though vary considerably in price; (d) the environment studied (fig 1) showed wide variations in their use, with differences in rates of prescription per inhabitant and month, adjusted for the age and sex of the population, of up to 7 times for piroxicam, 18 for tenoxicam, and 22 for diclofenac.
The effectiveness and safety of piroxicam, tenoxicam, and diclofenac used to treat arthrosis was analysed in light of the following measures of results: decrease in pain, residual pain, pain with movement, pain the day after onset of treatment, appearance of adverse effects, interruption of treatment because of adverse effects, suspension of treatment because of subjective ineffectiveness, subjective effectiveness for the doctor, and subjective effectiveness for the patient. The search of the literature was based on the review of pre-existing systematic reviews of the effectiveness of the NSAIDs in the Cochrane Library, using two complementary search strategies (appendix 1). Only reviews meeting the following criteria were accepted: (1) they included only randomised, double blind clinical trials; (2) they included the active drugs that were the subject of this analysis; (3) they analysed at least one of the measures of results cited; (4) they achieved a minimum of 15 points out of a possible 28 on the scale used for quality analysis (appendix 2). Of the three reviews found, one was selected for the study and the other two were excluded. Appendix 3 provides the references of the reviews and, when applicable, the criteria used for exclusion.
The systematic review led to the main conclusions that there are no differences between the three drugs studied, except for a lower rate of interruptions of treatment with tenoxicam compared with piroxicam. In light of these conclusions, during the outreach visit, diclofenac was recommended over the other drugs because it is cheaper, and when an oxicam was required, piroxicam was recommended, unless there was a risk of suspending treatment. The comparative information was summarised in a printed brochure, similar to those given out by pharmaceutical companies (available from the authors upon request), and was distributed during the visit. The visit emphasised the evidence based methodology followed, the conclusions reached after the review, and the recommendations formulated above (table 1).
The experimental intervention consisted of holding a group educational outreach session with the GPs of each of the primary care teams. During the session, oral and visual information was presented about the effectiveness and safety of the NSAIDs reviewed. The printed brochure was distributed. Diclofenac was recommended as the anti-inflammatory of choice to treat arthrosis and, when an oxicam was required, piroxicam was recommended as the best alternative. The visits were held in each primary care centre and were led by the primary care pharmacist for the area of Teruel, who had been previously trained in the principles of evidence based medicine (EBM) and the basic tools for the dissemination of this information. The GPs comprising the placebo group participated in a conventional educational session about NSAIDs, and did not receive printed support material. The primary care area pharmacist also conducted these sessions, although before she received training in EBM, and without her having any knowledge of the results of the systematic review. The control group received no information during the period of the study. The intervention period lasted less than four weeks.
Measurements of results
The main outcome was defined as the detection of changes in prescription patterns for the active drugs, and other related drugs, measured as the proportional change in the number of packages prescribed during the six months before intervention and the six months afterwards. We expected to observe a decrease in the prescription of the drugs that had not been recommended, and an increase in the prescription of those that had. A secondary outcome was the average cost per package of NSAIDs during the same before and after intervention periods, as the drugs that were recommended were cheaper. The data used in the follow up analysis of the intervention were obtained for each GP on a monthly basis, through the Pharmacy Information Service of the Area Directorate, which registers information about all the prescriptions issued in the province.
Other variables that were analysed were the percentages of retirees and women, the average number of medical visits per inhabitant per year, the average number of patients seen by each doctor per day and each doctor's previous prescription pattern, calculated as the average number of packages prescribed during the past six months and the average cost per prescription over the past six months.
The first analysis was a comparison of the basal characteristics of the three groups using the χ2 tests for qualitative variables and Snedecor's F test for quantitative variables. To analyse the main outcome, the relative reduction (RR) in the number of prescriptions of each type of NSAID was calculated as the average number of packages of each drug prescribed per doctor and per month during the six months after the intervention, minus the average number of packages prescribed per doctor, per month during the six previous months, divided by the latter. Significant differences were established in function of the confidence intervals of 95% (95%CI) of this reduction. The calculation of the 95%CI values was based on eight people (the primary care teams) in each one of the groups.
Because prescriptions of NSAIDs show seasonal behaviour, changes in costs per prescription (as a secondary variable) were also evaluated with a determinist time series analysis. The object of this analysis was to find a function to serve as the analytical representation of a set of data that evolves over time with reference to an imaginary line. Of the functions available, the S or Gompertz curve was selected. Its analytical expression, similar to lineal regression, is: y=exp(a+(b/t)); where a: the constant for the model, b: the slope y t: time. This model best adjusts to the empirical data for the 12 months before intervention, calculated with the r2 coefficient of determination and Snedecor's F test.14 A time series adjusted curve of the cost per prescription for each group and its confidence intervals was thus plotted, and the post-intervention changes registered in the groups were projected on this adjusted curve. The reduction in the cost per prescription (cost of average package during the six months after the intervention, minus the cost of average package during the six previous months, divided by the latter) between the periods prior and post-intervention was also calculated for each of the groups, with their respective confidence intervals of 95%.
Table 2 shows the basal characteristics of the experimental, placebo, and control groups. There were no significant differences between the groups in terms of the percentage of retirees or women, average number of patients seen by each doctor each day, the average number of visits per inhabitant per year, or the average cost per prescription. However, there were significant differences in the number of packages prescribed per month in the six months before intervention, which was higher in the experimental group compared with the control group. Participation in the group sessions was 81.2% of the doctors in the experimental group and 70.4% in the placebo group (p=0.34).
Table 3 shows the changes in the number of packages prescribed. The three groups showed a reduction in the number of prescriptions (−15.8% in the experimental group and −14.1% for the other groups, with no statistically significant differences between the groups). This change was accompanied with changes in the mix of therapeutic drugs. The main differences between the experimental and the control groups were observed in the reduction in the use of drugs that had not been recommended in the intervention. A decrease of 25.5% in meloxicam and 22.6% in tenoxicam was observed in the experimental group compared with reductions of 1.2% and 14.4% respectively in the control group. The placebo group showed results that were midway between the other two groups, with less marked differences that were not significant when compared with the others.
Figure 2 shows the evolution of the cost per prescription of NSAID in the three groups during the 24 month observation period, together with the adjustment lines and the confidence intervals of the overall series. The coefficient of determination of the adjustment curve was 0.69 (F:49.00; p<0.001). The evolution of the cost per prescription was within the confidence limits predicted by the adjustment in all the groups and, although the experimental group alone showed a dramatic change after the intervention, occupying the lower confidence limit of the adjustment line, and remaining within this range until the end of the experiment, the differences in the average cost per prescription were not significant. The percentage reduction was 1.91% (95%CI: −0.33% to −3.49%) in the experimental group and 0.16% (95%CI: −0.27% to 2.93%) in the placebo group. In the control group, a relative increase of 1.76% (95%CI: 0.35% to 3.17%) was observed.
Educational outreach visits seem to be an effective approach to modifying health professional behaviour, especially prescribing, but there is little evidence about which key characteristics of outreach visits are important to their success.
The literature reflects considerable variation in the effectiveness of the different educational strategies used, and there is no evidence supporting the superiority of any above the others.
This randomised study suggests that evidence based educational outreach visits are incrementally more effective than conventional educational sessions, which are, in turn, incrementally more effective than no intervention at all.
The results of this study show that prescription patterns of NSAIDs changed in the experimental group along the lines suggested by the content of the educational outreach visit. This change was significant compared with the control group, but not when compared with the placebo group, which participated in a conventional educational session. The change was not accompanied by differences in the total volume of NSAIDs prescribed (the intervention did not consider the need for the indication for NSAIDs, but the choice of specific NSAIDs once the indication had been established), nor did it translate into a significant reduction in the average cost per prescription for NSAIDs despite the fact that the analysis suggests that costs did tend to decrease.
These results are consistent with the literature3,9 and suggest that evidence based educational outreach visits are incrementally more effective than conventional educational sessions, which, in turn, are incrementally more effective than no intervention at all. It is not clear, however, whether the differences observed between the experimental and the placebo groups are attributable to the type of intervention, or to the fact that the experimental group received a brochure providing additional written reinforcement. In fact, the literature indicates that the effectiveness of the different educational strategies is highly variable, and there is no direct evidence of the superiority of any of them over the others.9 However, it has been seen that the more intensive strategies combining different approaches tend to yield better results.
Certain limitations are inherent in a study of this nature. The first is the potential failure of the randomisation process, or other selection biases. Although this cannot be entirely discarded when the number of items to be randomised is small (as in this case, 24 centres into three groups), it is unlikely to have occurred, as no differences were registered—except for one—in the basal variables that affect the volume of prescriptions (table 1), nor were differences registered in the overall relative increase of packages prescribed (−15.8% to −14.1% and −14.1% for the experimental, control and placebo groups respectively). Additionally, the selective change in prescription patterns observed, where there was a decrease in the prescriptions of the drugs that had not been recommended, while volumes for the others either remained stable or increased slightly (though not significantly), cannot be accounted for by evoking the differences detected in the overall basal prescription volumes between the groups, the only variable that showed any differences when basal values were compared.
Other possible threats to the internal validity of a study of this design are phenomena of maturation, regression to the mean pre-intervention score, and the misclassification of the effect. The possible influence of situations not related to the intervention was mitigated by conducting the visits in a short period of time, in fact in a span of less than a month. All of the primary care teams report to a single area healthcare directorate, which was blind to the randomisation of the centres into the three groups, and which did not conduct any other differential strategies during the period in question. Regression to the mean pre-intervention score would not explain changes in the type of active drugs prescribed and, furthermore, would be minimal in a time series adjusted as a moving average. Possible misclassification of the results of the intervention was minimal, as the information system loses less than 4.5% of the prescriptions for the whole healthcare area.
The conditions under which the intervention was conducted make it highly unlikely that the participants might have suspected which of the three groups they were in, and virtually rule out any other form of contamination. The healthcare centres are spread far apart, all the sessions were conducted in a short period of time and the entire healthcare area had already received all manner of materials from the pharmacy unit of the primary care directorate: all the doctors had been sent individualised information about their pharmaceutical consumption and expenditures, and had participated in other pharmaco-therapeutical sessions or had received pharmacology bulletins, etc. This made it improbable that the experimental or placebo sessions could have been perceived as exceptional situations, which would lead the participants to suspect that they were immersed in an intervention trial. For this reason as well, “novelty” or “social desirability” effects would not constitute alternative explanations for the results obtained. Finally, although the person who conducted the sessions was not blind to the randomisation of the groups, she had not participated in the systematic review of the evidence and was only given sufficient information and training to allow her to conduct the evidence based sessions once she had completed the intervention in the placebo group.
Limitations affecting the analysis of the data begin with the study's low statistical power (n=24 primary care teams distributed into three groups) to detect differences in the reduction in the percentage of prescriptions of the non-recommended drugs. Had the samples been larger, the differences detected between the experimental (−25%) and the placebo (−13%) groups, and the placebo and the control group (−1%) might have been significant. When analysing the cost per prescription, because of the seasonal nature of the use of the NSAIDs, a time series was chosen. More complex models, such as a random effects regression,15 may have been more suitable. However, as this variable was a secondary outcome, and the differences detected in the simple analysis were irrelevant, the more sophisticated analysis tools were considered unnecessary.
It is true that when evaluating the results of educational interventions it is difficult to isolate the effect of the intervention from other variables in the environment and the context of the study. Factors affecting outcome range from the choice and framing of the issue, to the way in which the visit is conducted, the critical analysis of the evidence, the preparation of materials for dissemination, empathy towards the session leader, and the overall climate within the organisation itself. None the less, the findings of this study indicate that the evidence based outreach session was more effective than no intervention at all, and provide support for the inclusion of evidence based interventions as part of the primary care doctors' routine, even in environments, such as the Spanish National Health System, where there are very few incentives, economic or otherwise, for changing behaviour.
APPENDIX 1 SEARCH STRATEGY
Search 1 Effectiveness
(#1 or #2)
(DRUG and THERAPY)
(OUTCOMES-RESEARCH and (SEE and OUTCOME-ASSESSMENT–HEALTH-CARE-*:ME))
(#3 and #4)
(#3 and #8)
(#9 or #10)
Search 2 Adverse effects
(ADVERSE and EFFECT)
(SIDE and EFFECT)
(PEPTIC and ULCER)
(GASTROINTESTINAL and HEMORRHAGE)
(DUODENAL and ULCER)
(STOMACH and ULCER)
(ESOPHAGITIS and PEPTIC)
(GASTROINTESTINAL and INJURY)
(GASTROINTESTINAL and BLEEDING)
(#1 or #2)
(#2 or #3) or #4) or #5) or #6) or #7) or #8) or #9) or #10) or #11) or #12) or #13) or #14) or #15)
(#1 and #17)
APPENDIX 2 CRITERIA TO EVALUATE THE ARTICLES RECOVERED
Existence of a research protocol. From 0 to 2 points. Categories: “No protocol”, “Essential parts are missing”, “Detailed protocol”.
Existence of search strategy or background. From 0 to 2 points. Categories: “Not specified”, “Insufficient”, “Specified”.
List of studies analysed and reasons for inclusion. From 0 to 2 points. Categories: “Not specified”, “List of studies”, “Reasons for inclusion”.
List of studies excluded and reasons for exclusion. From 0 to 2 points. Categories: “Not specified”, “List of studies”, “Reasons for exclusion”.
Analysis of quality of studies. From 0 to 4 points. Categories: “Qualitative”, “Quantitative”.
Comparability of studies. From 0 to 8 points. Categories: “Qualitative analysis of sources of heterogeneity”, “Statistical evaluation of heterogeneity”, “Suitability of the type of analysis”, “Analysis of sensitivity”.
Control of biases. From 0 to 8 points. Categories: “Analysis of blind studies”, “Analysis between two reviews and degree of concordance”, “Existence of different search strategies “, “Search for non-published materials”.
APPENDIX 3 STUDIES INCLUDED AND EXCLUDED AND REASONS FOR EXCLUSION
Studies used for review
Riedemann PJ, Bersinic S, Cuddy LJ, et al. A study to determine the efficacy and safety of tenoxicam versus piroxicam, diclofenac and indomethacin in patients with osteoarthitis: a meta-analysis. J Rheumatol 1993;20:2095–103.
Studies excluded from review
The Cochrane Musculoskeletal Review Group. A systematic review of randomized controlled trials of analgesia and anti-inflammatory therapy in osteoarthritis of the hip. The Cochrane library 1998 (issue 2). Aug 1997.
Reason for exclusion
The study compares drugs that are infrequently used and compares analgesics to NSAIDs.
The Cochrane Musculoskeletal Review Group. The comparative efficacy of non-aspirine non-steroidal antiinflammatoy drugs for the management of osteoarthritis of the knee. The Cochrane library 1998 (issue 2). Nov 1996.
Reason for exclusion
The study evaluates a drug that is not available in our environment (etodolac).
Thanks to Soledad Márquez and Ricard Meneu for their comments and suggestions.
Contributors EBD, MGM, and FPA were responsible for the conception and design of the study. EBD was responsible for the systematic revision, and MGM conducted the educational visits. EBD and SP were responsible for the analysis and interpretation of the data. All the authors contributed to writing the article and revising it for important intellectual content. All approved the final version.
Funding: this study was financed in part with FIS grant 98/0730 from the Fondo de Investigación Sanitaria and complementary financing from the Fundación Instituto de Investigación en Servicios de Salud.
Conflicts of interest: none.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.