Cave – Career Beginnings Impact Evaluation

Research Findings:

The Impact of Support for Disadvantaged High School Students on Later College Attitudes & Enrollment

Cave G. and Quint, J. (1990). Career Beginnings Impact Evaluation: Findings from a Program for Disadvantaged High School Students. Manpower Demonstration Research Corporation (MDRC), New York, NY.

Retrived on October 4, 2010 from http://www.aypf.org/publications/compendium/C1S40.pdf

Editor’s Note – An “impact” study such as this is one of the best forms of research possible because it isolates mentoring from other program factors and it uses two groups, one receives mentoring and one that does not. Also, it is quantitative (more objective) and not qualitative (more subjective). To clarify – This examines adult mentoring and other supports of high school Juniors.


INDEX


Summary

The essential research question was, “To what extent do mentoring and other support programs for disadvantaged high school youth result in improved college enrollment and attendance?”

The study participants were randomly assigned to either an experimental group which received the subject supports, or a control group which did not.

  • The experimental group was eligible for Career Beginnings, a program which included individual mentoring.
  • The control group was excluded from Career Beginnings but were free to participate in other services available in their schools and communities.

The study found that participants in various mentoring programs had both higher levels of college enrollment and higher educational aspirations than did the nonparticipants receiving comparable amounts of education and job-related services.


The Career Beginnings Program

Career Beginnings (CB), was piloted beginning in 1986 at Brandeis University – Center for Human Resources. CB, “identifies high school students with college potential who, because of average grades and disadvantaged backgrounds, might otherwise be unlikely to attend college.”

The CB program combines four types of support:

  1. career exploration and career plan development;
  2. educational enrichment;
  3. mentoring;
  4. employment.

Study Methodology

The Manpower Demonstration Research Corporation (MDRC) conducted the research study during 1987-1988, which was the second year of the CB pilot.

The study investigated CB program impacts in seven of the 24 CB sites.

Each site recruited approximately 200 juniors from local high schools who met the CB eligibility criteria. A total of 1,574 students were randomly assigned in equal numbers to experimental and control groups. There were no discernable differences between experimental and control group members at the start of the study.

The study findings were based on the responses of 1,233 youths to two follow-up interviews conducted one and two years after the random assignment.


Study Locations

The seven CB locations where the the study CB Program was examined were:

  • (NY)     Bronx Community College;
  • (Gary, IN)     Indiana University Northwest;
  • (Indianapolis, IN)     Butler University;
  • (Jacksonville, FL)     Jacksonville University;
  • (Rochester, NY)     University of Rochester;
  • (Santa Ana, CA)     Rancho Santiago Community College;
  • (Youngstown, OH)     Youngstown State University

Study Participants

Between 1987 and 1988, participants for these seven sites were part of a control / experimental groups evaluation to study the effects of CB. Participants were high school juniors who had substantially exceeded minimal academic requirements, ranked in the middle of their class academically, and demonstrated personal motivation and commitment beyond just school activities. By design:

  • at least 50 percent of participants at each site were economically disadvantaged
  • at least 80 percent were from families where neither parent had a college degree
  • at least 45 percent were male.

Findings

Statistically significant data showed that, compared to the control group, CB participants:

  • had a 9.7 percent increase in the rate of college attendance in the post-high school year, which was 53.2 percent versus 48.5 percent for the control group
  • more often started college “on schedule” in the Fall semester
  • worked less and earned less, which was expected due to time spent enrolled in college.
  • While all participants in the experimental groups showed statistically significant differences due to the CB Program, it was also found that locations that had “implemented the program most effectively” produced evidence of the largest impact. Therefore, “program implementation” information was more deeply studied for factors that could lead to defining best practices.
  • Insufficient numbers of staff at any location reduced that staff’s ability to fully implement the program as designed.

Features Common to All Studied CB Locations Were:

  • collaboration among a local college or university, public secondary schools, and the business community which provided the mentors;
  • jobs during the summer between the student’s junior and senior year;
  • monthly workshops and classes throughout the student’s 15 months of involvement, on a range of career development competencies and topics;
  • counseling to guide students as they made educational and career choices
  • adult mentors from the business and professional community who serve as role models and meet with youth one-on-one to help them plan for the future.

Developing a Systematic Approach to Implementation

Originally, CB allowed considerable flexibility across program sites and that led to marked variation in program implementation. Since locations that had “implemented the program most effectively” produced evidence of the largest impact, the study concluded that the extent and quality of “program implementation matter” and it suggested that CB Program leaders pursue “a more prescriptive approach” utilizing the best practices identified in the study.

Since insufficient numbers of staff proved to be a limitation on some locations’ ability to fully implement the program as designed, the study recommended “defining minimum staff requirements to ensure that programs have the personnel necessary to give a new program the attention it needs.”


Follow Up Research Studies

Upon conclusion of the CB pilot and the Cave-Quint study, 18 of the original pilot communities continued their CB efforts and 15 new communities were added to the program, all implementing the refinements in the model that resulted from the original study. Information on these later CB program versions from
after 1990 through 1997, has been provided by William Bloomfield of School & Main, the company handling management of the CB Program.

The sample has grown considerably from about 1,400 in the original study to more than 15,000 participants later (largely B/C grade average, first generation in college, at-risk). These more recent and larger studies found:

  • 95 percent of the CB Program participants were graduated from high school
  • 71 percent of the CB Program participants went on to college compared to 37 percent of students who went to college in the same demographic group nationally.

Conclusions From the Original and Follow Up Studies

The original CB results and more current good results are attributed to the following factors which were proposed as best practices.

1. Effective Implementation

CB worked to find the appropriate balance between broad flexibility and common goals, objectives and design. While sites have great flexibility in how they deliver services to students, years of experience have led to some consensus about critical, non-negotiable programmatic elements which are uniformly high
in quality and geared to achieve measurable outcomes. Three factors were key.

A. A high quality, very effective range of student services. (effective, comprehensive program parts);

B. Integration of those student services to achieve the “synergy” that makes the overall delivery of services even more effective;

C. Effective local program leadership – to orchestrate the efforts of and sustain local community partnerships, and provide an explicit focus on management and operations to ensure full implementation of the program design.

2. Broad-Based and Productive Partnerships

Several factors were found to be essential:

A. Engaging in active, shared leadership a broad base of local stakeholders (K-12 and higher education, business partners, and local government) creates a truly systemic partnership

B. Partners roles should include management and implementation activities.

C. The resulting partnership should have”less of a programmatic and more of a community development character,” which increases support for education reform and improves school-to-career system building and seamless delivery of services to students.