Wanted: Teachers and Parents to Evaluate Anti-bullying Programs

Do Current Anti-bullying Programs Work?

Recently, two studies (1, 2) reviewed the effectiveness of anti-bullying programs. Smith and colleagues synthesized the results of 14 evaluation studies of whole-school anti-bullying programs, examining rates of victimization and bullying reported by children. The results indicated that only one program yielded outcomes that were consistently positive. The remaining programs that were evaluated yielded little or no improvement. Similarly, Vreeman and Carroll (2) reviewed the outcomes of 26 programs (some of which were also reviewed by Smith et al. (1)). Only three of these programs yielded consistent reductions in bullying and victimization, eight yielded some modest positive outcomes, and ten yielded no positive results at all.

These reviews underscore several important points that should be of concern to educators, parents, and researchers alike. The first is that there has been very little research on anti-bullying programs generally, and more evaluation studies are urgently needed. Second, there is currently only limited evidence that anti-bullying programs are effective in curbing this problem. Third, the positive results obtained in several studies suggest that prevention programs have the potential to significantly reduce bullying, but more information is needed to understand how they can be improved and made more effective.

I believe that educators and parents can and should take an active interest in evaluating their school’s anti-bullying program. Evaluation requires some additional time and effort but is undoubtedly a sound investment. Here are a few of the benefits of evaluation:

• Schools learn whether or not their programs are achieving desired outcomes and, by extension, if their resources are being wisely allocated.
• Despite the good intentions of those involved in implementing prevention programs, research tells us that many are not implemented as intended (3). Instead, interventions are commonly adapted, corners are sometimes cut out of necessity, and some staff members are disinterested or even resistant to doing their part to ensure program success. Evaluation allows school personnel to account for the quality of the program as implemented and subsequently fine-tune the implementation process.
• People tend to act differently when they are being observed (as in the context of an evaluation) and typically in a manner that improves their performance. Schools can take advantage of this so-called “Hawthorne effect” to maximize the chances of program success.
• Program evaluation represents a learning opportunity for schools, as the people involved learn about the program and how it works. This knowledge often improves the quality of program implementation, which in turn leads to better outcomes.
• Evaluation provides opportunities for critical, constructive reflection that are characteristic of schools with healthy climates. A positive climate in the school may be the key ingredient to making bully prevention programs successful (4).

Basic Evaluation Techniques

To streamline the work involved in the process of examining an anti-bullying program, it is useful to set up a committee to organize and run the evaluation. Ideally, the committee has representation from the entire school community, including teachers, administrators, parents, and students. The evaluation process is guided by objectives, such as the two listed below:

1. Verify that the program is being implemented as planned: To address this objective, a checklist can be prepared that includes all of the activities comprising the anti-bullying program. Staff involved in implementing the program can be asked to indicate if and how often each activity is completed. This checklist should be filled out several times throughout the year (e.g., once a month or once a semester). By aggregating this information, a good picture of how a program is being implemented will emerge. Schools wishing to investigate aspects of program implementation in more depth can also organize debriefing meetings with staff and students to probe the successes and challenges of their specific anti-bully program.
2. To determine if the program is achieving its intended results: To address the second objective, the evaluation committee must first decide what outcomes are expected from the program. Examples of possible results include: increases in anti-bullying attitudes, increases in positive student behavior, reductions in bullying and victimization, and improvement in school climate. It is important that the selected outcomes be feasible and realistic given scope and content of the program and the time allotted for implementation. The next step is to identify and find questionnaires that measure these outcomes. There are many such questionnaires in circulation, and often they can be procured easily for free (e.g., via the web) or for a modest fee from the publisher. The questionnaires should be administered on the following schedule:

a. Just before the program begins (pre-test)

b. Immediately after the program ends (post-test)

c. Several months (and up to a year) after the post-test (follow-up)

Finally, the committee reviews the evaluation data and makes recommendations for improving program implementation and outcomes.

Educators and parents can play a critical role in making anti-bullying programs more effective and ultimately reducing bullying among children. Initiating local evaluation projects to understand and improve programs is an effective way to reach the goal of making schools safe havens for learning and healthy development.

References

1. Smith, J. D., Schneider, B. H., Smith, P. K., & Ananiadou, K. (2004). The effectiveness of whole-school anti-bullying programs: A synthesis of evaluation research. School Psychology Review, 33, 548-561.
2. Vreeman, R. C., & Carroll, A. E. (2007). A systematic review of school-based interventions to prevent bullying. Archives of Pediatric and Adolescent Medicine, 161, 78-88.
3. Dane, A., & Schneider, B. H. (1998). Integrity in primary prevention programs: Are implementation effects out of control? Clinical Psychology Review, 18, 23-45.
4. Smith, J. D. (2007). School Climate and Bonding: Pathways to Resolving Bullying. Queens Education Letter, Fall/Winter 2007, 10-11. Accessed online at http://educ.queensu.ca/alumni/letter/issues/QueensEducationLetter_FallWinter07.pdf on January 17, 2008.

Resources

1. Olweus Bullying Prevention Program: www.clemson.edu/olweus/index.html
2. Measure of school climate: See Table 1 in Gallay, L., & Pong, S. (2004). School climate and students' intervention strategies. Accessed online at http://www.pop.psu.edu/socresp/quebec1.pdf on January 17, 2008.
3. Measures for bullying: Ontario ministry of Education (2007). Safe schools: Preventing bullying. Accessed online at http://www.edu.gov.on.ca/eng/safeschools/bullying.html on January 17, 2008.

Biographical statement

J. David Smith, PhD, is an Associate Professor of Educational Counselling at the Faculty of Education, University of Ottawa. His research examines the effectiveness of school-based anti-bullying programs, and in particular how school climate affects the outcomes of these programs. He can be reached by email at: David.Smith@uOttawa.ca.