Supplement articleEvaluating the impact of interventions in the Multisite Violence Prevention Study: Samples, procedures, and measures
Introduction
T he Multisite Violence Prevention Project (MVPP)'s evaluation of the GREAT (Guiding Responsibility and Expectations in Adolescents Today and Tomorrow) parent, teacher, and student interventions represented a significant advance in the evaluation of universal and targeted prevention programming. As noted by Henry et al.1 in this supplement, the 2 × 2 experimental research design in which 37 schools were assigned randomly to conditions allowed us to compare the effects of a universal intervention (i.e., classroom curriculum and teacher training/support groups) to a targeted intervention (i.e., multifamily groups) on outcomes such as aggression, school climate, and school norms among sixth-grade students at the school level. The design also permitted tests of the potential benefits from simultaneous implementation of universal and targeted interventions relative to implementation of either in isolation.
The assessment procedures and measures used in the evaluation of the GREAT programs were designed to be consistent with the focus of the project's interventions and research design.1, 2, 3, 4 Measures were selected to assess the impact of the interventions on: (1) the overall population of sixth-grade students as represented by the cohortwide sample—a random sample of sixth graders in each school; (2) high-risk youth and their parents, as represented by students at each school meeting criteria for the targeted program and their parents; and (3) teachers, as represented by the total sample of sixth-grade core academic teachers (e.g., math, social studies, language arts, or science) in each school. Intervention effects on each of these domains were assessed using multiple sources of data that addressed change at the individual level and overall school norms and climate.
Measures included in the outcome battery assessed behavioral outcomes, as well as mediators and moderators of change. In terms of behavioral outcomes, the interventions were expected to reduce a range of aggressive behaviors including relational, nonphysical, and physical aggression. Thus, our measures included victimization from, and perpetration of, each of these forms of aggression in peer relationships as well as within dating relationships. The measures also included mediating variables representing factors that the interventions were expected to influence directly, which in turn would reduce risk for aggression. They also included moderators, or factors that had the potential to enhance or reduce the success of the intervention. These mediators and moderators were assessed in a manner that was most appropriate for each type of intervention. For example, potential mediators of change for the classroom and teacher interventions included problem-solving skills and self-efficacy to use positive relationship skills. For the family intervention, mediators of change included parental disciplinary practices and parental involvement in their child's school. Potential moderators of change included beliefs and norms about violence, deviant peer involvement, and self-efficacy to use positive relationship skills.
This paper describes three important aspects of the multisite evaluation protocol. First, we describe the different samples and sources of data collected for each sample. We also review other data being collected, including measures of school variables and process/fidelity data. Second, we explain our data processing and collection procedures. This includes a discussion of institutional review board (IRB) procedures and coordination of these procedures across the sites. In the last section, we describe the steps taken to finalize the instruments and summarize the final set of measures used in this project.
Section snippets
Samples and types of data
In this section, we describe the three samples of participants, the sources of data used to assess the impact of the interventions on each sample, and the timing of data collection. We also review other types of data being collected, including interviews with the school principal, an assessment of the school environment, and process/fidelity data.
Data management and collection procedures
In this section, we discuss the multisite data center and its role in managing the data for the sites. We also discuss the computer-assisted survey interview (CASI) that was used to collect most of the self-report data. Our description of data collection procedures focuses on the student and parent surveys because they were the most complicated. We also describe IRB procedures and coordination of these procedures across the multisite project.
Finalizing the measures
Before describing the final measures, we briefly describe the procedures used to select these measures. This iterative process involved close collaboration among the work groups developing the universal and targeted intervention components, the work group focusing on the assessment procedures, and the principal investigators. The initial list of constructs and recommended instruments proposed by each site was based on their experience and reflected their preferences for specific measures and
Final measures
Table 4, Table 5, Table 6 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38 provide a complete listing of the scales used in the final student, teacher, and parent surveys, respectively, including references for the original source and data on internal consistencies. Each measure is also identified by its function as an outcome variable or potential moderator or mediator.2, 3, 4
Summary
The MVPP assessment procedures not only provided data necessary to evaluate the behavioral outcomes of interest and to understand the processes through which the interventions influence aggressive behavior, but also addressed gaps in the field of violence prevention. Tolan and Guerra41, 42 noted the need to understand how violence prevention efforts impact multiple levels of influence on youth behavior (i.e., individual, family, school) and the effect of prevention programming on different
Acknowledgements
The authors gratefully appreciate the efforts of the site-based research coordinators: Yvonne Wasilewski and Ann Skinner (Duke University), Ann Green (Virginia Commonwealth University), Franklin Gay (The University of Illinois, Chicago) and Tracy Elder (University of Georgia). We would also like to acknowledge the many school personnel that were of tremendous assistance in facilitating data collection efforts. Finally, we especially acknowledge the youth and parents who participated in the
References (42)
- et al.
Program integrity in primary and early secondary preventionare implementation effects out of control
Clin Psychol Rev
(1998) - et al.
A comparison of paper vs. computer-assisted self interview for school, alcohol, tobacco, and other drug surveys
Eval Program Plann
(2000) - et al.
Prevention of delinquencycurrent status and issues
Appl Prev Psychol
(1994) - et al.
Progress and prospect in youth violence-prevention evaluation
Am J Prev Med
(1996) - Henry DB, Farrell AD, Multisite Violence Prevention Program. The study designed by a committee: design of the Multisite...
- Meyer AL, Allison KW, Reese LE, Gay FN, Multisite Violence Prevention Program. Choosing to be violence free in middle...
- Orpinas P, Horne AM, Multisite Violence Prevention Program. A teacher-focused approach to prevent and reduce students'...
- Smith EP, Gorman-Smith D, Quinn WH, Rabiner DL, Tolan PH, Winn D-M, Multisite Violence Prevention Project....
- et al.
Behavior assessment system for children
(1992) - et al.
A typology of parent-rated child behavior for a national U.S. sample
J Child Psychol Psychiatry
(1999)
The study of implementationcurrent findings from effective programs that prevent mental disorders in school-aged children
J Educ Psychol Consult
Implementing audio-CASI on Windows platforms
Comput Human Behav
Adolescent sexual behavior, drug use, and violenceincreased reporting with computer survey technology
Science
The impact of T-CASA interviewing on reported drug use among men who have sex with men
Subst Use Misuse
The structure of self-reported aggression, drug use, and delinquent behaviors during early adolescence
J Clin Child Psychol
Relational and overt forms of peer victimizationa multi-informant approach
J Consult Clin Psychol
The aggression scalea self-report measure of aggressive behavior for young adolescents
J Early Adolesc
Character and community developmenta school planning and teacher training handbook
An evaluation of Safe Dates, an adolescent dating violence prevention program
Am J Public Health
Children's responses to peer conflicts involving a rights infraction
Merrill-Palmer Q
Teen conflict survey
Cited by (66)
Young Adolescents' Digital Technology Use, Perceived Impairments, and Well-Being in a Representative Sample
2020, Journal of PediatricsCitation Excerpt :School belonging (n = 2104) was assessed with the 6-item Psychological Sense of School Membership23 self-report scale of school membership on a scale from 0 [not at all true] to 5 [very true] (eg, I feel like a real part of my school; people at my school are friendly to me; α = 0.84). Conduct problems (n = 2103) were assessed using the 26-item Problem Behavior Frequency Scale24 of behavioral aggression and violence in the last 30 days on a scale of 0 (never), 1 [1-2 times], 2 (3-5 times), 3 (6-9 times), 4 (10-19 times), and 5 (≥20 times) (eg, in the last 30 days, how many times have you …skipped school; stolen something from another student. This scale was converted into a count of reported problems (binary for each item rated >1 then summed).
Prevention of disruptive behavior problems in children
2018, Developmental Pathways to Disruptive, Impulse-Control, and Conduct DisordersDigital Location Tracking: A Preliminary Investigation of Parents’ Use of Digital Technology to Monitor Their Adolescent’s Location
2023, Journal of Family PsychologyTrajectories of Perceived Technological Impairment and Psychological Distress in Adolescents
2023, Journal of Youth and AdolescenceThe Social Cognitions of Victims of Bullying: A Systematic Review
2022, Adolescent Research ReviewBeliefs as Mediators of Relations Between Exposure to Violence and Physical Aggression During Early Adolescence
2022, Journal of Early Adolescence