The original Mentor Competency Assessment (MCA) scale was published as “The mentoring competency assessment: validation of a new instrument to evaluate skills of research mentors” in 2013 by Fleming et al. in Academic Medicine, a Journal of the Association of American Medical Colleges. The 26-item scale measured six sub-domains of mentor competency: maintaining effective communication, aligning expectations, assessing understanding, addressing diversity, fostering independence, and promoting professional development. The original MCA was used in the national longitudinal Enhance Diversity Study (Building Infrastructure Leading to Diversity (BUILD) Faculty Annual Follow-up Survey 2017-2108 and 2019; National Research Mentoring Network (NRMN) Annual Follow-up Survey 2016-2017, 2017-2018, and 2019) as part of the evaluation of the Diversity Program Consortium, supported by the National Institutes of Health (NIH U54GM119024).
How is academic self-efficacy related to biomedical career outcomes? Stemming from Albert Bandura’s work on self efficacy (1977) and later work on social cognitive theory (Bandura, 1991), self-efficacy was developed to explain how people think, motivate themselves, and ultimately how they behave-- including how long they will persist in the face of obstacles or challenging situations (Bandura & Adams, 1977). Defined as “a person's beliefs concerning his or her ability to successfully perform a given task or behavior” (Bandura, 1977; Betz & Hackett, 1981, p. 400), an individual’s perceived self-efficacy is based on four dimensions: personal mastery experiences (i.e., experiencing successes and failures), vicarious experiences (i.e., observing others succeed such that it strengthens the belief in oneself to also be able to succeed), verbal persuasion (i.e,. feedback and encouragement from others), and emotional state (i.e., level of anxiety and stress).
How is self-efficacy--specifically with respect to scientific research-- related to biomedical career outcomes? Albert Bandura’s theory of self-efficacy (1997) has been well-researched and has expanded to examine specific domains such as the science disciplines. Defined as “a person's beliefs concerning his or her ability to successfully perform a given task or behavior” (Bandura, 1977; Betz & Hackett, 1981, p. 400), an individual’s perceived self-efficacy is based on four dimensions: personal mastery experiences (i.e. experiencing successes and failures), vicarious experiences (i.e. observing others succeed such that it strengthens belief in oneself to also succeed), verbal persuasion (i.e. feedback and encouragement from others), and emotional state (i.e. level of anxiety and stress). The theory was developed to explain how people think, motivate themselves, behave, and persist in the face of challenges and anxiety-provoking situations (Bandura & Adams, 1977). Self-efficacy is particularly vital when considering the challenges involved in conducting scientific research, the persistence required to achieve the training milestones needed to advance in a biomedical career, and the current culture of science, technology, engineering, and mathematics (STEM) that discourages (Robinson, McGee, Bentley, Houston, & Botchway, 2016) more often than motivates and encourages student growth, sometimes leading to students switching into non-science majors and career aspirations (Seymour & Hewitt, 1997).
While there are numerous definitions of mentoring, it is often described as an extended relationship with a more senior individual who has knowledge and/or experience that can be used for career success by a more junior individual. Unlike teaching, which has a well developed set of skills and approaches that have been systematically studied, mentoring tends to rely on the highly individualized practices of mentors which are only now being systematically studied for effectiveness (NASEM, 2019).
The practice of mentoring new scientists along their career paths has been integral to cultivating the scientific workforce. In an effort to address the gap between research on effective mentorship and how mentorship is practiced in science, technology, engineering, and math (STEM) disciplines at colleges and universities, the National Academies of Sciences, Engineering, and Medicine (NASEM) developed a report to summarize what is known about effective mentoring relationships, particularly for STEMM1 disciplines (NASEM, 2019). While the report affirms the importance of mentorship and provides recommendations based on current mentorship research, there is still much to learn about the role of trainees’ perceptions of mentorship, and to what extent satisfaction with the quality of mentorship matters in comparison to other aspects of mentorship, such as the quantity of mentors or a specific mentoring structure (Eby et al., 2013)
Social integration in higher education has been defined as the students’ degree of social and psychological comfort with their campus environments, association with or acceptance by affinity groups, and a sense of belonging that provides the security to join with others in common causes, whether intellectual or social (Braxton, 2000). Sense of belonging, defined as a student’s subjective sense of “fitting in” (Spady, 1970; Hurtado & Carter, 1997) is one is one aspect of social integration that focuses on the psychological dimension of integration. Perceived sense of belonging within the university (STU-5), as well as within the research community (STU-6), are two of the student-focused hallmarks of the Diversity Program Consortium (DPC).
The National Institutes of Health (NIH) funds training programs to increase the numbers and skills of scientists who obtain NIH research grants, but few programs have been rigorously evaluated. The sizeable recent NIH investment in developing programs to increase the diversity of the NIH-funded workforce, implemented through the Diversity Program Consortium (DPC), is unusual in that it also funds a Consortium-wide evaluation plan, which spans the activities of the 10 BUilding Infrastructure Leading to Diversity (BUILD) awardees and the National Research Mentoring Network (NRMN). The purpose of this article is to describe the evaluation design and innovations of the BUILD Program on students, faculty, and institutions of the 10 primarily undergraduate BUILD sites.
National reports call for improving America’s leadership in scientific research, accelerating degree attainments, and diversifying the scientific workforce to foster innovation. However, slow progress and persistent disparities across growing U.S. populations are evident on key science workforce indicators, from degree attainment to career achievements. The purpose of this article is to provide a conceptual basis and overview of a national effort funded by the National Institutes of Health (NIH) that advances inclusive science practice and systemic change. We introduce the context, features, and rationale that drive practice and evaluation in the Diversity Program Consortium (DPC) approach, which is an experimental program to implement and evaluate evidence-based and novel practices to expand and diversify the biomedical workforce.
The National Institutes of Health (NIH)-funded Diversity Program Consortium (DPC) includes a Coordination and Evaluation Center (CEC) to conduct a longitudinal evaluation of the two signature, national NIH initiatives - the Building Infrastructure Leading to Diversity (BUILD) and the National Research Mentoring Network (NRMN) programs - designed to promote diversity in the NIH-funded biomedical, behavioral, clinical, and social sciences research workforce. Evaluation is central to understanding the impact of the consortium activities. This article reviews the role and function of the CEC and the collaborative processes and achievements critical to establishing empirical evidence regarding the efficacy of federally-funded, quasi-experimental interventions across multiple sites. The integrated DPC evaluation is particularly significant because it is a collaboratively developed Consortium Wide Evaluation Plan and the first hypothesis-driven, large-scale systemic national longitudinal evaluation of training programs in the history of NIH/National Institute of General Medical Sciences.
The NIH-funded National Research Mentoring Network (NRMN) aims to increase the representation and success of underrepresented groups (URGs) in biomedical research by enhancing the training and career development of individuals from diverse backgrounds, communities, and cultures. The national scope of NRMN, its wide array of innovative programs in mentor and mentee matching and training across the career spectrum, requires a collaborative evaluation strategy that addresses both internal and external evaluation needs. Due to the variety of programs implemented for each target group, the NRMN program is responsible for its own process and short-term outcome evaluations and the national Coordination and Evaluation Center (CEC) is responsible for assessing the medium and long-term effectiveness of the implemented strategies and program sustainability. Using a collaborative, utilization-focused evaluation framework, both internal NRMN evaluators and the CEC are working to translate findings into information that can be used to make both short term and long-term decisions about the efficacy and reach of the NRMN model. This important information can then inform efforts to institutionalize the current programs and potentially replicate them elsewhere.
Science identity represents the extent to which students conceive of themselves as scientists and is one of the hallmarks of the NIH Diversity Program Consortium (DPC).a Data from the 2017 Student Annual Follow-up Survey (SAFS) indicates that BUILD students in biomedical majors have a higher average science identity score than non-BUILD biomedical students at the same institutions. The higher science identity among BUILD students occurs for most racial/ethnic groups and for all items in the science identity scale. Given the importance of science identity to continuing on the biomedical career pathway, these results suggest that BUILD students are more likely to persist in science majors and careers than other biomedical science students at their institutions.
A major NIH initiative is underway to determine the most effective ways to engage students from diverse backgrounds in biomedical research, and to prepare students to become potential future contributors to the NIH-funded research enterprise. The 10 grantees vary widely, from Historically Black Colleges and Universities (HBCUs), where most biomedical graduates are from underrepresented groups, to large public schools with moderate representation of underrepresented groups among their biomedical graduates. Graduation data from four years pregrant and two early years of the BUILD program show a general trend of constant proportions of underrepresented minority (URM) students graduating in biomedical majors, with a few institutions showing a rapid growth in the numbers that reflects overall increases in their number of students graduating with biomedical degrees over that time. Since few graduating students would have been involved in the new programs in the period shown, these trend data are best considered baseline or preintervention information that can be used in future years to help assess the institutional impact of the BUILD grants.
The National Research Mentoring Network (NRMN) provides researchers across all career stages with mentorship and professional development programming that emphasizes the benefits and challenges of diversity, inclusivity and culture (https://nrmnet.net). Individuals register on the NRMN online portal to use virtual mentoring, access online resources, or sign up for a wide variety of training events. When enrolling through the online portal, all registrants are required to identify as a mentor or mentee, regardless of how they use NRMN.
Within the biomedical disciplines, faculty-student mentorship is a key component for success. Mentors provide support, motivation, and advice to students that increase their social capital in research, as well as their skills and identities as scientists. Increasing mentoring self-efficacy is one of the hallmarks (FAC-3) that serves as an intermediate outcome for the NIH Diversity Program Consortium (DPC).1 Data from the 2016–2017 HERI Faculty Survey2 finds that biomedical faculty respondents at the 10 BUILD sites report having higher mentoring self-efficacy scores if they are involved with the BUILD program than if they are not. This suggests that BUILD students are likely to have access to higher quality mentoring than other biomedical students.