Assessing the Impact of Mentoring – A Paradigm Shift

By Dr. Joseph Pascarelli


T.S. Eliot the poet, in his work Four Quartets, warns us,   “We had the experience, but we missed the meaning.”    Are YOU too busy DOING Mentoring to ASSESS the impact of that mentoring? Then this article is for YOU.

A Need to Shift from Advocacy to Assessment

There’s a new frontier in mentoring for those of us who are committed to the concept and practice of mentoring. No longer do we need to focus as heavily on advocacy – beating the drums and selling the concept. The world is well aware of the value of mentoring. The rich variety of sectors embracing mentoring well attests to this. Simply check the Internet, and more importantly, the increasing number of colleagues who attend the IMA’s annual conference!  Mentoring has become pervasive.

There is, however, an important and more timely role for us to address and that is to assist others in assessing the impact of mentoring relationships. The purpose of this short article is to deepen our understanding and identify ways in which impact needs to be measured.

Is Mentoring Really Making a Difference?

How often do we hear from colleagues variations of, “But how do we know if the resources and energies we invest in mentoring are really making a difference ?”

Assessing or evaluation mentoring is a matter of documenting human growth in all of it’s applications:

  • Whether it relates to the undergraduate proteges of instructor-mentors in four Hong Kong universities
  • The clusters of elementary and middle school proteges who are mentored by community volunteers in New York City
  • The apprentice carpenters (proteges) in Oregon who are being mentored by senior journeyman
  • The middle and high school level female proteges who are being mentored by community volunteers as part of a pregnancy-prevention initiative in Michigan
  • The early elementary level male students being mentored by the retired group of male businessmen at Gardine School in St. Thomas, Virgin Islands
  • The at-promise minority youth in Syracuse schools being mentored by local business people; or…
  • The women and children who are living in battered shelters who, as proteges, are mentored by community volunteers in Washington.

The present challenge is to assist those responsible for these programs, the coordinators or managers, in identifying and designing the appropriate ways to measure the effectiveness of these programs.

What Changes Happen When a Protege Grows?

Specifically, people in mentoring programs need to know how they can demonstrate that growth is occurring on the part of proteges in mentoring relationships.

A study of the knowledge base in this area suggests that, as a result of effective mentoring, proteges essentially become empowered. What specifically does that look like?

  • Proteges begin to increase their self-confidence and trust in themselves.
  • Proteges who are supported by effective mentors increase their capacities
    • to make thoughtful decisions
    • to work through problem resolution weighing potential consequences of actions, and…
    • consequently to make better value choices.
  • Thus, they become more inner-driven and self-reliant.
  • Proteges learn the value of human connections and that some other persons in this world are interested and supportive of them.
  • They understand the difference between being helped as contrasted with being rescued. Mentors do the former; professionals frequently address the latter.

Healthy mentoring relationships give meaning to Donne’s universal message “no man is an island” or Buber’s views on humanity’s responsibility to each other-to give a “Yes” to the other and acknowledge his/her existence. Proteges essentially want their Yes-affirmation that they belong and are connected. They learn about hope and the power of envisioning possible futures. This is the essence of effective mentoring.

Methods for Demonstrating Protege Growth

What are the more appropriate ways for measuring growth?

  • In attitudes (e.g., feelings of improved self-concept and self-worth, visioning) and…
  • In cognitive skills (e.g., problem solving, decision making, personal goal-setting and planning)

Among the techniques that are more appropriate to use are:

a) interviews

b) focus groups

c) self-inventories and assessment

d) rating scales and rubrics

e) observations.

A. Unstructuredinterviewsthat use probing questions and open-ended statements and that resemble conversations will capture the richness of the protege’s experiences in the mentoring relationship.

B. A skilled facilitator who used focus groups will sustain a dialogue among a group of proteges that can
reveal the extent to which these proteges are making significant personal meaning (intentional reflecting and learning) out of their experiences.

C & D. Rating scales or inventories
are based on self-perceptions. These tools will require respondent-proteges to spend time reflecting, to examine their journals, logs, or diaries and to both discover and then express the journey they have taken.

E. Observations are admittedly more difficult to conduct, but they will enable a keen participant-observer to capture the power of the mentoring experience through field notes and anecdotes.

If carefully designed using observable (behavioral) indicators, these techniques will yield data that will capture the extent to which the critical concepts like ego-strength, self-concept, locus of control, or self-determination occurs in proteges. More quantitative approaches to evaluation simply will not be able not to capture and demonstrate that such changes are happening.

Evaluating mentoring programs and specifically, the growth of proteges can best be done by “insiders” (participant-observers) rather than by external and “objective” evaluators. The practice is based on listening, probing, and integrating.

Mentoring is contextual and relational. That which occurs is personal, rich, and deep. Admittedly, this is yet another paradigm shift-this one is moving from a perspective based on “clipboards, statistics, and logic” to one that values “dialogue or voice, descriptions, and stories.”

Much like the hokey-pokey (based on Jungian psychology), one must put the “whole self in” to play the game. And so it is with learning about changes that proteges are experiencing.

Dr. Joseph T. Pascarelli is Assistant Professor at the University of Portland. Dr. Pascarelli has been a member of the IMA Board of Directors since March 2001 and is a Past President of the IMA.

Editor’s Note – The reader is also referred to other articles on this web site which discuss the CBAM Stages of Concern as a method of planning, facilitating, and documenting human growth.