Sourcebook | Action Planner | Task Force Planner & Guide | Highlights | Publications CD
Evaluation: Who, What, How!
By David S. Anderson, Ph.D.
Director, Center for the Advancement of Public Health George Mason UniversityAssessment and Evaluation. How often we hear these words and told how important these issues are. Further, we increasingly hear more about the need to do good assessments, to gather information about our projects, and to obtain some insights about whether what we did made a difference. And, within this context, we have feelings about the difficulty of accomplishing this. We feel overwhelmed, panic, challenged about what we know about our students, and more.
This article identifies some details and strategies to take some of the "sting" out of the process, and to re-assert the importance of evaluation. The recommendation cited in the Promising Practices: Campus Alcohol Strategies Sourcebook states: "Conduct and use ongoing evaluation. The importance of evaluation could fill an entire subset of recommendations." Indeed, entire books and resources are written about evaluation and assessment, and are valuable for incorporation in any campus program. (The article represents the first in a series of articles elaborating upon the ten recommendations summarized in the Sourcebook published by the Promising Practices: Campus Alcohol Strategies project. It is perhaps symbolic that the series of articles begins, rather than ends, with the recommendation about evaluation).
The first step in evaluation, whether a student leader, BACCHUS and GAMMA affiliate member, staff advisor, or other concerned individual, is to clearly identify what you want to accomplish. In any project being implemented, clearly specify what you want to see different as a result of what you do. This is particularly important because you are trying to make a difference in the lives of others, and you do have heartfelt commitment to doing something meaningful. Thus, first think about what you want those affected by the project to know, to feel,or to do; what do you want to see different with them? This clarification helps you take the major first step to accomplishing what is known as outcome evaluation. Once you know what you want to accomplish, then you can build the measures that help monitor this.
What often happens in evaluation activities is that so much evaluation emphasizes the process evaluation, the other major type of evaluation effort. What process evaluation does is very important; it examines how the activity went, how it was received, what could be done to improve it, and how many people were served by it.
With evaluation activities, it is important to have both outcome evaluation and process evaluation. The outcome evaluation monitors whether our efforts made a difference, and the process evaluation is helpful in knowing what went well and what needs to be improved. We need both of these, because one without the other doesn't tell us the whole picture. For example, if we know that an event was well received, that it got good reviews, that the attendance was high, and that things went smoothly, this is great! It tells us that, to repeat it or do another event, we should take some lessons from this to have another successful event. If something wasn't particularly smooth, that we can make changes to adjust it. This is helpful for other campuses as we share what we're doing, so they can replicate our good works. What it doesn't tell us is whether the event actually made any difference in people's lives.
That's where we need outcome evaluation; this monitors what we attempted to accomplish. If we set out to change some attitudes and to increase participants' knowledge, we can check this out with a variety of measures, such as a quick pre-test before the event and one right at the end, and then comparing the results to see if any changes occurred (and if they went the way we wanted).
Other brief tips and insights include:
1. With each activity, aim for parallelism between (a) what you want to accomplish; (b) what you do; and (c) what you evaluate. Once you know what you want to accomplish, the other two aspects can fall into place more easily.
2. As a student, staff or faculty leader with efforts on the campus, your role with evaluation is to get it done, or see that it gets done, not necessarily to do it yourself. It's important to not panic with the evaluation "assignment"; simply have a commitment to seeing that it gets done.
3. Think about your own campus' resources which can be helpful in accomplishing the evaluation. Ask your chief student affairs officer who might be available to assist with this; resources include an institutional research office; faculty members in psychology, education,health sciences, sociology, business, public policy, and related areas; or a graduate or undergraduate student who could provide the expertise and assistance. Think creatively, and realize that you don't have to find everything in one place.
4. Identify the many resources that already exist; you don't have to "reinvent the wheel." Review some of the numerous pockets of excellence cited in the Sourcebook: refer to these approaches and call/write/E-mail them with questions or to request copies of their instruments, questions, and protocols. Think about using resources such as the Higher Education Center for Alcohol and Other Drug Prevention or the National Clearinghouse on Alcohol and Drug Information, or picking up the telephone and calling someone who has written an article in a professional journal and asking some key questions.
5. Be creative. Blend ideas together to best suit your campus' needs by taking one approach and incorporating another issue. Blend, mix, adjust, adapt, adopt!
6. Keep your expectations reasonable. Your campus effort will probably not make all the changes you would like to see. Realize that your effort can accomplish something, but most efforts are simply small steps in the process of building a larger outcome, each of which are important, since without them the final outcome would never be reached. Thus, our evaluation should try to measure these small steps as it keeps an eye on the longer term result.
7. For each activity or initiative, try to get some measure from both a process and an outcome perspective. At a minimum, gather some brief feedback and data. Use a brief survey form; this may be standard for all of your events!
8. Write down, now, a couple a ideas that you intend to do in the next week. This may be to search out people on campus, looking up some evaluation strategies in the Sourcebook or other resources, bringing the issue to the attention of your BACCHUS and GAMMA chapter or other campus organization, or even getting copies of this article made and sharing it with others!
In summary, the issue of evaluation is, indeed, complex. However, there are some steps that are reasonable for you to take, regardless of your role with the campus. Your commitment to trying to better capture some of what is happening regarding your programs and initiatives will be helpful, both for your students and your campus. And, with your creativity and commitment,your assessment and evaluation strategies can help other campuses in their efforts to measure and monitor, and ultimately redirect, their own campus strategies.
David S. Anderson, Ph.D. is an Associate Professor at George Mason University's Center for the Advancement of Public Health. Promising Practices: Campus Alcohol Strategies is based at George Mason University in Fairfax, VA. Project Co-Directors are GMU's Dr. Anderson and Gail Gleason Milgram, Ed.D. of the Center of Alcohol Studies, Rutgers University. Funded by a grant from The Century Council, Promising Practices: Campus Alcohol Strategies hosts a web site at http://www.promprac.gmu.edu; project staff can be contacted at 703-993-3697 or caph@gmu.edu.
If you would like to order, at no cost, any of the materials featured on this website, please e-mail your request including your name and mailing address. For additional information about Promising Practices: Campus Alcohol Strategies, please call 703-993-3697 or e-mail us.