© By Barry Sweeny, 2010
The following list presents the basic program evaluation model which I use to guide my own consulting and program evaluation work. Of course, in practice, the steps are more complex than is clear here. Never-the-less, I share this with you to aid you in evaluating your own mentoring program. The concept I have in mind as I suggest this is that using this sequence will assure you that your evaluation has the right steps in the process. Other pages elsewhere on this web site will help you be sure that you also use best practices as you do each step.
The original ideas for this model are loosely derived from the research and work of both Tom Gusky and Donald Kirkpatrick. See below for citations. The model has been developed through use and refinement over about 20 years of work in curriculum, staff development, with grants, and many other kinds of programs. It has been refined and adapted to the point where I feel it represents a “best practice” model.
Please note – The “Tools” menu has a version of this list which has added columns for check offs and notes.
Steps in the Process:
- Set the Purpose and the Goals for the evaluation
- Identify the audiences for the conclusions and recommendations
- Define the indicators of success for each audience
- Check the alignment (relevance) of each indicator to program goals
- Establish the evaluation parameters – what you will and will not do
- Determine the scope of the evaluation process
- Organize the indicators by data type:
- Reactions (survey attitudes)
- Learning (self-assessment/test)
- Behavior (observe extent of use)
- Needs Assessment (obstacles to use? needs?)
- Results (effectiveness, gains)
- State the evaluation questions
- Refine the evaluation questions for:
- Validity & Fairness
- Select how to collect the data:
- Design/select the instrument(s)
- Check the need to validate instrument(s)
- List the evaluation implementation plan:
- List the activities
- Set the time line
- Decide the persons responsible
- Estimate the cost of the evaluation
- Gain approval to do the evaluation plan
- Collect the data
- Analyze and interpret the data
- Evaluate the evaluation
- Report the conclusions & offer recommendations
- Develop improvement priorities (focus?) and targets (how much improvement?)
- Develop an improvement action plan
Gusky, T. (1998). “The Age of Our Accountability”. Journal of Staff Development. Fall, 36-44. National Staff Development Council. This is not the original source I used, but it is a good and more recent one.
Kirkpatrick, D. (1975). “Evaluating Training Programs”. Madison, WI. American Society for Training and Development.
Kirkpatrick, D. (1977). “Evaluating Training Programs: Evidence Vs Proof”. Training and Development Journal. 31, (11), 9-12.