Meta-Analysis – Mentoring in 3 Settings

Summary – Does Mentoring Matter? A Meta-Analysis of Mentoring Outcomes in Three Settings

Summary by Barry Sweeny, December, 2010.

This is a summary of the following unique and very significant research report.

Eby, L.T., Allen, T.D., Evans, S.C., Ng, T. and DuBois, D. (2008). Does Mentoring Matter?  A Multidisciplinary Meta-Analysis Comparing Mentored and Non-Mentored Individuals. Journal of Vocational Behavior. 2008 April; 72(2): 254–267.

Retrieved online at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2352144/


INDEX

  1. Introduction – Problem and Goal
  2. The Meta-Analytical Study
  3. Differences in Youth, Academic Student, and Workplace Mentoring
  4. Eligibility Criteria for Inclusion in the Study
    1. Excluded Studies
    2. Selected Studies
  5. Meta-Analysis Procedure
  6. The Hypotheses To Be Researched
  7. Findings
    1. Differences by Type of Mentoring
    2. Possible Explanations for These Differences
  8. Guidance on the types of outcomes to expect mentoring alone to impact
  9. Editor’s Notes
  10. References

Introduction – Problem and Goal

Research into mentoring is typically done within a specific setting. The result of this is that there are no quantitative, and therefore objective reviews of the mentoring knowledge base as a whole. This is especially unfortunate given that the same basic motivations and process apply to all those contexts. We all know that when a more experienced person (mentor) takes an interest and invests time, caring, and wisdom in a less experienced or disadvantaged person (protégé), the protégé will benefit. This certainty begs the questions:

1. What desired outcomes for protégés can be expected from mentoring regardless of setting?

2. What desired outcomes for protégés are best realized in any specific setting?

The lack of cross disciplinary research in mentoring results in little cross-disciplinary communication among mentoring scholars. The result of that is a lack of clearly defined, universally applicable best practice guidelines for mentoring practice or mentoring program design.

This example of mentoring research undertakes the answering of these two vital questions. In research language,

this meta-analytic study will examine youth, academic student (higher education), and workplace mentoring to determine the overall effect size associated with mentoring outcomes for protégés.


The Meta-Analytical Study

This study was a comprehensive, multidisciplinary meta-analysis with the objective of answering the question, “Looking across different areas of mentoring research, does mentoring matter, and if so, how much?”


Differences in Youth, Academic Student, and Workplace Mentoring

Although similar in some respects, youth, academic student, and workplace mentoring also differ.

  • The Protege’s Developmental Stage – Every effective mentor assesses protégé development and readiness and designs mentoring to suit that stage of growth. Yet what is appropriate mentoring at one developmental stage is not at others.
  • Different Goals – Mentoring at different developmental stages typically serves different goals. For example, it makes little sense to discuss and set career goals when mentoring an elementary aged child. hence, youth, higher education student, and workplace mentoring are likely to have different foci. Specifically:
    • Youth mentoring is often targets reduction of risky behavior or improvement of social and academic success.
    • Academic student mentoring tends to target student retention, academic performance, and adjustment to college life and responsibilities.
    • Workplace mentoring generally targets enhancement of employee personal and career development, skills, productivity, and effectiveness.

Eligibility Criteria for Inclusion in the Study

The initial search process yielded 15,131 articles and reports with potential contributions. To be considered for inclusion an accepted study had to:

1. be written in English
2. compare mentored and non-mentored individuals on one or more specific outcomes (e.g., academic success, drug use, work attitudes)
3. quantify the relationship between mentoring and the outcome using a statistic that could be converted to a correlation coefficient
4. focus on traditional one-on-one, non-parental mentoring relationships.
5. For intervention studies, mentoring had to be the sole or primary intervention   (See Editor’s Note #1**)
6. To compute a meta-analytic correlation on a factor, at least three studies of that factor were required


Excluded Studies

Studies were excluded if they focusing exclusively on:

  1. peer, group / team, or reverse mentoring
  2. parents as mentors, professional caregivers or specialists as mentors, and social support from teachers;
  3. new teacher induction programs, on-the-job training, or internship programs, since these studies do not necessarily involve one-on-one mentoring.  (See Editor’s Note #1**)

Selected Studies

One hundred and twelve studies and research reports met all of the eligibility criteria. Three studies included multiple samples, for a total of 116 independent samples for the meta-analysis.  (See Editor’s Note #2**)


Meta-Analysis Procedure

  1. The 15,131 articles were screened by two of the study authors.
  2. The first and second authors were responsible for coding all studies included in the meta-analysis so that factors related to the research question were identified and isolated.
  3. These two coders independently double-coded articles until they reached over 90% agreement.
  4. Correlations were judged to be significant at alpha=.05 when the 95% confidence interval did not include zero.

The Hypotheses To Be Researched

Mentoring is associated with positive …

1: behavioral outcomes.
2: attitudinal outcomes.
3: health-related outcomes.
4: relational outcomes.
5: motivational outcomes.
6: career outcomes.


Findings

Hypotheses 1–6 were each supported. Regardless of the meta-analytic method used (fixed- or random-effects), mentoring was significantly related to favorable behavioral, attitudinal, health-related, interpersonal, motivational, and career outcomes. These results demonstrate that mentoring is positively
associated with a wide range of favorable outcomes, although the effect size is small for some of these.

Some differences were also found across type of mentoring. Generally, larger effect sizes were detected for academic and workplace mentoring compared to youth mentoring.

  1. The largest effect sizes were between mentoring and increases in:
    1. helping others
    2. school attitudes
    3. career attitudes.
  2. The smallest (but still statistically significant) effect sizes were between mentoring and:
    1. decreased psychological stress and social deviance (when estimated using fixed-effects only)
    2. increased career recognition and success,, and positive self-perceptions.

The meta-analysis revealed several patterns across different types of mentoring.

  1. Behavioral outcomes, all three types of mentoring demonstrated significant effect sizes for performance.
  2. Academic student mentoring was more highly related to performance than was youth or workplace mentoring.
  3. Both youth and academic mentoring were significantly associated with reduced withdrawal behavior, whereas workplace mentoring was not.
  4. Analysis of mentoring and helping others showed: significant effect for workplace mentoring
    1. no significance for youth mentoring.
  5. Attitudinal outcomes, all of the effect sizes were significant. However,
    1. academic student mentoring was a stronger effect than youth mentoring.
  6. Health outcome – decreased psychological stress,
    1. the workplace mentoring effect size was significant whereas the youth effect size was not.
  7. Interpersonal relations, mentoring in both settings was significant, but,
    1. the effect size was stronger for workplace mentoring than for youth mentoring.
  8. Motivational involvement,
    1. the effect sizes were significant for academic and workplace mentoring
    2. the effect size was not statistically significant for youth mentoring.

Overall Findings

Our findings are generally consistent with previous reviews focusing on just youth, academic student, or workplace mentoring.  Four conclusions were reached from the results.

  1. Mentoring is significantly and favorable correlated with a wide range of protégé outcomes.
  2. Although the overall effect sizes are all small, mentoring appears to be more highly related to some protégé outcomes (e.g., school attitudes) than to others (e.g., psychological stress & strain).
  3. There is mixed evidence for other unexamined factors effecting some mentoring-outcome relationships.
  4. There is tentative evidence of differences in the extent to which mentoring is associated with some outcomes for youth, academic student, and workplace relationships.

Differences by Type of Mentoring

Some interesting differences in effect sizes were found across the three mentoring settings. The absolute value of the effect sizes were distributed as follows.

THE RANGE OF EFFECT SIZES FOR EACH SETTING
SETTING >
youth mentoring
workplace mentoring
academic student mentoring
EFFECT SIZE >
.03 to .14
.03 to .19
.11 to .36
youth mentoring
workplace mentoring
academic mentoring
.01-.03
.03-.11
.11-.14
.14-.19
.19-.36
.36-.40

This pattern suggests that

  • academic student mentoring has stronger associations with outcomes than does youth mentoring
  • workplace mentoring is somewhere in between.

Possible Explanations for These Differences

The context in which these different types of mentoring occur impacts the effects.

  • Youth typically are mentored because they are “at risk” for behavioral, social, or academic problems. Yet these may be difficult to overcome with mentoring alone (which is the focus of the study). In fact, there is some evidence that youth mentoring leads to greater benefits when integrated with other support services (Kuperminc et al., 2005).
  • It may be more difficult for mentors to offer focused and tailored guidance given that youth may have many needs especially when compared to the typical protégé within an academic or workplace setting.
  • Academic student mentoring is often focused on a behavioral outcome such as performance because adolescents who have made it to higher education have already surmounted or never faced some of the other obstacles and they are functioning at a higher level.
  • Academic student mentoring is often considered to be a core component of addressing an institution’s mission
  • Higher education mentors may be better equipped to provide mentoring given the nature of their career.
  • Typically, the persons who mentor youth or serve as informal mentors within the workplace setting do so on a volunteer basis with little or no training.

Guidance on the types of outcomes to expect mentoring alone to impact:

  1. Youth mentoring is most likely to affect school attitudes and least likely to affect the performance, psychological stress, or the motivation / involvement of protégés.
  2. Higher education student academic mentoring has the most affect on improving performance, attitudes toward school, and decreasing withdrawal behavior.
  3. Workplace mentoring provides larger gains in enhancing helping behavior, situational satisfaction, and attachment, and interpersonal relationships whereas smaller gains are expected for enhancing job performance and deterring withdrawal behavior.

Editor’s Notes

1. The exclusion in this study of support programs other than just mentoring allows the researchers to later claim that the effects measured can be solely attributed to mentoring and no other support program or factor. For the sake of this research, this exclusion is important and necessary.

However, readers should not assume that this endorses use of mentoring without other forms of protégé support. In fact, this very study states in it’s Findings, that there is research that suggests otherwise.

Further, this editor has never seen any research to indicate that mentoring should be or is best done by itself, devoid of other protégé supports. On the contrary, there are significant numbers of research studies to document, for programs with goals for:

  • skill development
  • performance improvement
  • increased desired results,

that it is necessary to use a comprehensive program model in which mentoring is one central, strategy out of several, such as:

  • protégé orientation
  • protégé training
  • protégé peer support activities
  • protégé observation of expert peers, and
  • protégé professional development goals and action plans.

2. For the sake of brevity, this summary will not include the 112 citations for the research studies examined by this meta analysis. Readers who desire that detail may visit the cited web link near the title of this summary.


References

Kuperminc, G. P., Emshoff, J. G., Reiner, M. M., Secrest, L. A., Niolon, P.H., & Foster, J. D., (2005). Integration of mentoring with other programs and services. In Dubois, D.L., & Karcher, M.J. (Eds.), Handbook of youth mentoring (pp. 314-333). Thousand Oaks, CA: Sage.