Evaluating & Improving EXISTING Programs

By Barry Sweeny, 2003

The”overview” presented on this page is provided just to give you the big picture, so that as you get into the greater details of the process, you have a rough idea of how all the parts of the process fit together, and so you know where you are as you move through the process.

Presented below is a chart of the entire process for development and refinement of a NEW  mentoring program. However, if you have an EXISTING mentoring program, you will only need to do certain parts of this process, such as those itemized just below. The actual ones that are needed depend on what you have already in place and running well, those which are in place but not running well, and those that are missing at this time. Therefore, it’s best to print it out and review the whole list. Then mark it accordingly as a plan.

Typically, existing programs need to do:

  • A Needs Assessment – Steps #7 & 8
  • A Program Evaluation – Step #14
  • Development of a Proposal for Increased Program Funding – Step #20

The best practice process for improvement of any parts of an existing program is much more complex than described here.

  • Best practice guidance for every step is found on other web pages where you can learn the details for the more complex steps in the process.
  • There are a few basic assumptions on which this model is based, such as WHEN and HOW to do certain steps. These are clarified in other pages so you can decide if they are YOUR assumptions.

As always, you really should consider working with a mentoring expert who can consult with you during your refinement process and BE YOUR MENTOR. Think of the trial and error learning you can avoid and the time you can save! Click here to access a list of possible mentoring consultants.

Getting Started 1. Create a program planning group (PPG)

  • Identify needed institutional & constituency partners
  • Identify, contact & recruit representatives
2. Discuss and define desired results, and write program purpose statement – See an example
3A. Define/select the conceptual framework underlying participant development & improvement of results3B. Develop or select a Best Practice Framework for a Mentoring & Induction Program
4. Define links among other existing improvement initiatives and the mentoring program
Research 5.Search in the relevant literature for program & participant best practices linked to desired results

  • Collect articles and research reports
  • Read articles and write summaries of content
6. Presentation to PPG of the best practices linked to desired results
Needs 7. Assessment of local needs

  • Protege needs
  • Organizational needs (retention, improved performance, etc.)
  • Accountability needs & issues
8. Assess what each participant group already knows and can do related to expectations, role & tasks, once those are defined. in step 12. E.g. Mentor Task Self-Assessment
Program Goals & Strategy 9. Review information & select appropriate strategies

  • Discuss research information & data
  • Reach conclusions about which needs to address
  • Reach conclusions about program strategies & participant practices that address selected needs
Time Line 10. Set measurable long-term program goals and objectives for each of the first three years
11. Discuss & set a program implementation time line
Roles & Tasks 12. Define participant expectations, roles, and tasks

  • Protégés
  • Mentors Roles & Tasks – Buddy or Guide Roles
  • Managers & Supervisors as various levels
Incentives 13. Discuss and define necessary participant incentives & recognition – Some ideas?
Program Evaluation Plan 14.Plan the program evaluation

  • Develop parameters & standards for program evaluation
  • Define evaluation questions to be answered
  • Define audiences for evaluation data, conclusions & recommendations
  • Discuss & select measurable indicators to evaluate that link to program goals
    • Completion of planned program activities & extent of program implementation
    • Completion of planned participant activities
    • Attainment of program goals and objectives
    • Professional growth of participants & leaders (eg. changes on conceptual framework)
    • Extent to which identified needs are met
    • Extent of change in professional practices by participants and leaders
    • Changes in protégé learning, performance & productivity
  • Check alignment of evaluation indicators to program goals & objectives
  • Define data collection process
    • Identify target populations from which to collect data
    • Identify tools needed to collect data and need to validate tools
    • Define data collection, analysis, and reporting time line
    • Select person(s) responsible for implementation of plan
  • Develop targets for improvement for each indicator
Selection & Matching 15. Discuss, define & plan participant selection (& matching?) processes & criteria -Example
Training 16. Discuss, define program leadership needs & plan program leadership training & support needed
17. Discuss, define & plan participant training & support needed
18. Discuss, define & plan other needed program activities & their coordination
Budget Planning 19. Define resources and budget needed to support the program
Program Proposal 20. Prepare a program proposal providing:

  • participant & organizational needs
  • research on other’s mentoring programs and their effects, best practices, lessons learned, etc.
  • program goals & description
  • program implementation plans
  • program evaluation plans
  • needs for resources & budget to support the program and attain desired results
Seeking Support 21. Communicate with and seek program support from key decision makers
22. Use your program proposal to seek grant funding for initial implementation
Implementation 23.Implement plans according to the implementation time line and available budget
Data Analysis 24. Collect needed participant data & analyze program data patterns
25. Reach conclusions about progress toward goals & other program indicators
26. Reach conclusions about needed program revisions
Refinements 27. Implement needed program revisions
Report to Decision Makers 28. Report to decision makers & participants on program changes and participant progress