© 2002, Barry Sweeny
It is easier to make this mistake than one might imagine. You carefully create data collection tools which you later find are too big, with too many items, which are too complex and confusing, and which discourage responses.
It is also very easy to generate more data than you actually can effectively tally and use.
You need to carefully plan and execute that plan to avoid both of these conditions.
The goal of this article is to ensure that:
- The evaluation process produces data which actually helps the program accomplish its goals;
- You do not collect more data than can be effectively used;
- A high percentage of participants are willing to sincerely think about and complete the instruments you design.
Checking the alignment of the indicators you have selected against the goals and objectives of the program is essentially a comparison of these three lists to ensure that each is linked to the others. For example, any indicator that does not clearly relate to at least one goal is a candidate for tossing out, as it is not going to provide data related to any goals, and is therefore, superfluous and unneeded.
Here is a matrix that you can use to make such a comparison. To use it:
1. Enter the key words for each of your program goals where the word “Goal #1”, etc,. are.
2. Enter your program’s annual objectives under that where the word “objectives”, etc. are.
3. Enter the evaluation indicators you plan to use where the numbered indicators are.
4. Consider how clearly each indicator is aligned to and addresses each objective for each goal.
5. Place a “Y” or the word “Yes” in the appropriate cell when they are clearly linked, and a “N” or “No” when they clearly are not.
6. Place a “?” when the alignment is possible or questionable. These indicators either need to be rewritten to better assess the objective and goal, or they need to be discarded.
The matrix below has some cells completed to illustrate how such responses might appear. The result allows you to determine which indicators are well aligned, which need better alignment or to be dropped, and which would work if improved some. You could write an action plan using those same three sections.
|GOAL #1||GOAL #2|
|INDICATORS /||Objective 1A||Objective 1B||Objective 2A||Objective 2B||Objective 2C|