The EVAL Framework : Developing Impact Evaluation Scholars

The complexities of food, agriculture, natural resources, and human sciences (FANH) programs and projects require faculty to write and secure funding in addition to mastering skills such as evaluation competencies that integrate abilities in quantitative and qualitative research methods and evaluation theory and practice. The EVAL Framework was developed to advance skill development among FANH graduates to include these competencies and increase the pipeline of students who have the essential skills needed to advance FANH initiatives and priorities. The EVAL Framework includes four primary constructs: (a) Evaluation, (b) Value, (c) Active and Experiential Learning, and (d) Leadership. The purpose of EVAL is to build relationships with untapped FANH fields to develop a pipeline for graduates to become evaluation leaders for advancing food and agricultural sciences. This experiential learning and development model focuses on foundational and enrichment experiences, through formal coursework, project-based learning, and contextually rich environments. Mentoring, individualized development plans, scholarly learning communities, collaboration, transferable skills, and career planning and guidance activities are integrated into the EVAL framework to enhance skill development among EVAL scholars.


Introduction and Problem Statement
The complexities of food, agriculture, natural resources, and human sciences (FANH) programs and projects require scholars to not only master the ability to write and secure funding but also to develop mastery of evaluation competencies that integrate abilities in quantitative and qualitative research methods, evaluation theory, and practice. The United States Department of Agriculture (2018) recommended scholars and practitioners should be better prepared to administer quality program evaluations to assist policymakers in constructing the bestinformed decisions. However, these skills alone do not provide meaningful approaches for continuous improvement given the complexities of interdisciplinary issues in FANH. Analysis, cultural adaptability, life-long learning, ethical decision making, creativity, teamwork, communication, and leadership are essential skills needed for post-graduate success (Succi & Canovi, 2020). Fostering effective essential skills in graduate education holds the promise of developing graduates who are able to thrive in a world in which teamwork and collaboration are omnipresent (Britton et al., 2017). These essential competencies, coupled with evaluation competencies, create evaluation scholars and practitioners who will help fill the identified gaps in the food and agricultural sciences.

EVAL Framework
The authors created the EVAL Framework to develop students' competencies to be effective food, agriculture, natural resources, human sciences, and public health evaluators in addition to content experts in their disciplines. The EVAL Framework includes four primary constructs: (a) Evaluation, (b) Value, (c) Active and Experiential Learning, and (d) Leadership. The purpose of the EVAL framework is to connect scholars within a variety of FANH fields and further develop a pipeline for graduates to become leaders capable of advancing food and agricultural sciences. This experiential learning and development model focuses on foundational and enrichment experiences through formal coursework, project-based learning, and contextually rich environments.
The purpose of this article is to describe in detail (see Figure 1) the EVAL framework juxtaposed to the authors' experiences in implementing the framework within the context of a doctoral degree program in Agricultural Leadership, Education, and Communication disciplines. The article concludes with a vignette in the conclusion that highlights seminal takeaways from implementing the EVAL framework.

EVAL Framework: An Experiential Approach
Evaluation EVAL's inclusion of evaluation methodologies are central to the framework. Evaluation is a process that critically examines a project or program in terms of outcomes, outputs, and impacts. It involves collecting and analyzing information about a program's activities, characteristics, and outcomes. The purpose of evaluation is to make judgments about a program, to improve its effectiveness, and/or to inform programming decisions (Patton, 2014).
Evaluation studies may take place in both formal and informal settings, ranging in size and scope from a pilot study to a complex project that addresses different topics, involves hundreds of individuals, and includes a variety of methodologies (Ibarra-Sáiz et al., 2020). The EVAL students gain knowledge and skills in a variety of assessment and evaluation methods for social and behavioral science within FANH disciplines.
Evaluation skill development includes formative, summative, and process assessment. Formative assessment results are used in the formation and revision process of a social or behavioral phenomenon for continuous improvement (Rossi et al., 2019). Summative assessment is used for the purpose of documenting outcomes and judging value, such as providing feedback about the quality of a program, reporting to stakeholders, and granting agencies, producing reports for accreditation, and marketing the attributes of a program (Rossi et al., 2019). Most FANH projects are rarely exclusively summative in practice, and usually contain some aspects of formative assessment. Process assessment begins with the identification of project milestones to be achieved, activities to be undertaken, products to be delivered, and/or projected costs likely to be incurred while attaining the project goals. Rossi et al. (2019) indicated a process assessment determines whether the project has been on schedule, deliverables produced, and cost effectiveness. The degree of difference from the expected process is used to evaluate success.
In addition to the aforementioned skill set, EVAL students learn about theoretical and practical ethical practices for conducting evaluative research for building/maintaining community trust. Rather than the "giver" or "gatherer" of information (expert), the role of an EVAL practitioner is to share facilitation roles to equally value local knowledge (empowerment). From expert to empowerment is manifested through interactive data collection techniques, like transect walks, group discussion, matrices for ranking needs and other active ways of engaging the community. It parallels the qualitative/action research paradigm (Guba & Lincoln, 1981) and the empowerment of local communities with learners as co-creators of knowledge (Freire, 1970). EVAL is a credible tool for needs assessment and evaluation research in communities with low numeration and literacy skills. The specific skills to develop evaluation plans, strategies and tools builds upon the foundation coursework and active/experiential learning aspects of the EVAL model.

Values
EVAL aligns with identified approaches to ensure individual value and equity. According to Stufflebeam (2001), evaluations are grounded in clear and appropriate values, principles, attributes, or qualities held to be intrinsically good, desirable, important, and of general worth. Ensuring the next generation of evaluation scholars can apply competences in diverse settings within communities requires the ability to create, communicate, and disseminate culturally relevant results, while simultaneously considering societal values and equity.
Aligning individual values and realizing potential bias is at the core of program evaluation practice (Patton, 2021). EVAL students can investigate their personal values and learn how their values have the ability to impact evaluations. EVAL students learn strategies to minimize bias in evaluations through active and experiential learning methods. The American Evaluation Association's (AEA's) (2018) guiding principles emphasize the importance of an evaluators' obligation to not only become knowledgeable about other cultures, but also have the competence to evaluate the context in which the evaluator operates. As the U.S. continues to become more diverse, the field of evaluation has not yet followed. The lack of familiarity with other cultures affects the ability for an evaluator to effectively guide, conceptualize, analyze, and interpret an evaluation (Fitzpatrick et al., 2011). EVAL students engage and learn culturally responsive pedagogical teaching practices in the classroom. Culturally Responsive Teaching (CRT) is a learning centered pedagogy that recognizes students' cultural background, along with their past and current experiences (Ladson-Billings, 1994). The EVAL students take courses in which teaching strategies of CRT are taught and subsequently utilize these transferable skills in other settings.
Students engage in critical classroom discussions, dialogue, and are provided multiple enrichment experiences enabling them to utilize the skills and experiences they have learned to become culturally competent individuals. While becoming experts and leaders in the field, CRT will not only meet, but exceed the expectations of the AEA through the facilitation of constructive and culturally responsive feedback while conducting an evaluation. In this regard, the EVAL students can move the field forward and have a better understanding from multiple perspectives within the agricultural sciences, the field of evaluation, while immersing student in the discipline's values.

Active and Experiential Learning
EVAL involves active and experimental learning opportunities for impact scholars. Active learning involves fostering attitudes, developing and practicing skills, and promoting understanding of the concepts and models within the subject (Silberman, 2006). The EVAL framework is imbibed with experiences that provide active learning environments for graduate students, mentorship, formal courses, non-formal education programs, and faculty development academies. A foundation of active learning involves the implementation of experiential learning opportunities to enhance knowledge (Silberman, 2006).
Experiential learning has been identified as a tenet of agricultural education (Roberts, 2006). Kolb's (2014) experiential learning has four elements that indicate students learn best when actively engaged in the content versus merely observing. The first element is concrete experience and refers to students being involved in new experiences. The EVAL framework provides these new experiences to students by examining evaluation methodologies and instruments used by the EVAL team on current projects. Concrete experiences are also given, formally, in classroom settings via andragogical methods as well as informally during mentoring and individual development plans.
The second element is reflective observation. This refers to students being provided opportunities to reflect and critically think about what they learned in the concrete experience stage. Authors address this stage with EVAL students with pre and post reflections to mentor students as they develop valid instruments and strengthen transferable skills in a scholarly learning community. EVAL students also engage in reflection and critical thinking during their time in the scholarly learning communities.
The third stage focuses on abstract conceptualization and indicates the importance of students connecting increased knowledge to other situations (Kolb, 2014). The authors mentor EVAL students in applying information gained from reviewing evaluations by guiding students in building evaluation instruments for new assessments. CRT also enhances EVAL students' ability to connect values and cultural representation to the evaluation measures learned and applied through the program.
The fourth stage of Kolb's (2014) model is active experimentation. This stage focuses on using theories and models to solve problems. The authors address active experimentation by collaborating with EVAL students in developing evaluation plans based on models learned during the concrete experience stage. These active and experiential learning strategies align with the project objective to increase skills to promote a culture of learning for evaluation within an organization, to engage users and beneficiaries in evaluation processes and to broaden the use of evidence in decision-making.

Leadership
EVAL utilizes Brungardt's (1996) model of leadership development as the leadership construct. This model incorporates leadership trainings and workshops (specific skills), leadership education (formalized leadership theoretical knowledge), and leadership development (active learning application and reflection) to construct knowledge and make meaning from the formal and informal aspects of the program related to their role as a leader. By understanding the theoretical basis of leadership practice through formalized leadership education, then applying theory to practice, the EVAL framework engages scholars in sustainable leadership development. Specific leadership trainings and workshops where EVAL students can increase leadership skills include individualized development plans, leadership roles and teamwork seminars, and workshops within learning communities. Foundational EVAL courses integrate active and experiential learning with leadership theories using cultural competencies. Leadership theory and application (leadership development) creates scholars who not only demonstrate evaluation competence but know how to apply and lead in settings where complex problems need to be addressed (Hitt & Tucker, 2016). This requires the ability to work in teams, ethical decision-making, and intra and interpersonal communication.
EVAL students are encouraged to attend seminars that bring in international and diverse experts in emerging agricultural issues. Ethical decision making and problem solving is introduced in the Leadership in Organizational Culture and Ethics course and practiced by the students through case studies, evaluation projects with their principal investigator mentors, and experiential activities, including simulations (Boyd & Strong, 2020).

Enrichment Activities
The Academic advising, research training, and mentoring occur formally using the Kram (1985) model of mentorship as well as informally through synergistic activities. The following activities underpin EVAL: Individualized Development Plans (IDPs), Scholarly Learning Communities (SLCs), Mentoring, Institutional and International Collaborations, Transferable Skill Development, and Career Planning and Guidance. The IDP serves as the center of the experiential learning experiences by allowing benchmarking for each EVAL student to guide their progression of knowledge and skill attainment. The Center for Teaching Excellence at [University] provides access to an IDP education process model. The IDP encourages students to engage in critical reflection to enhance their experience.
Scholarly Learning Communities (SLCs) serve to create community among EVAL students, faculty, and fellow graduate students and provide a venue for professional development. This prevents isolation and builds collaboration. Monthly sessions focus on topics identified as needs during orientation and through the IDP. Mentoring has been proven to be critical in the development of graduate students as it encourages networking, provides support, and ensures proper guidance (Curtin et al., 2016). Each EVAL student is matched with a mentor who reflects their interests and long-term goals.
Institutional and international collaborations allow graduate students to see their work across multiple contexts to extend their networks. Application of essential skills, within varying contexts and with multiple collaborators, strengthens students' competencies (Jackson, 2010). Therefore, EVAL students are encouraged and supported in the attendance of relevant conferences and institutional visits. Collaboration is also encouraged during the engagement of students in the writing and implementation of evaluation plans.
Transferable skill development allows EVAL students to take the essential skills learned through formal and informal instruction and utilize them in their personalized contexts. Transferable skill development is accomplished via workshops targeted to meet students' needs. These workshops occur across the duration of the program are based upon the IDPs.
Career planning and guidance is critical for EVAL students to be successful upon graduation. Individual consultations take place between each EVAL student and appropriate faculty members. In addition, the Texas A&M University's Career Center is accessed for resources for further career development.

Competency Domains
Using the AEA Competency Domains, the authors aligned the EVAL learning outcomes with the AEA competencies and the planned foundational and enrichment experiences (AEA, 2018). The first AEA competency domain is professional practice. The competent evaluator acts ethically through evaluation practice that demonstrate integrity, respects people from different cultural backgrounds and indigenous groups, selects evaluation approaches and theories appropriately, uses systematic evidence to make evaluative judgments, reflects on evaluation formally or informally to improve practice, identifies personal areas of professional competence and needs for growth, pursues ongoing professional development to deepen reflective practice, identifies how evaluation practice can promote social justice and the public good, and advocates for the field of evaluation (AEA, 2018).
The second AEA (2018) domain is methodology. The competent evaluator identifies evaluation purposes and needs, determines evaluation questions, designs credible and feasible evaluations that address identified purposes and questions, determines and justifies appropriate methods to answer evaluation questions (e.g., quantitative, qualitative, and mixed methods). Evaluators identify assumptions that underlie methodologies and program logic, conducts reviews of the literature, identifies relevant sources of evidence and sampling procedures. Stakeholders are collaborators in evaluation design, implementation, interpretation, and reporting evaluation results. The evaluator, in the methodology domain, uses program logic and program theory as appropriate, collects and analyzes data using credible, feasible, and culturally appropriate procedures (AEA, 2018). Lastly, the evaluator identifies strengths and limitations of the evaluation design and methods, interprets findings/results in context, uses evidence and interpretations to draw conclusions, make judgments and provide recommendations.
The third domain is context. This domain focuses on understanding the unique circumstances, multiple perspectives, and programs being evaluated and their users/stakeholders (AEA, 2018). Context involves the program's site/location/environment, participants/stakeholders, organization/structure, culture/diversity, history/traditions, values/beliefs, politics/economics, power/privilege, and other characteristics.
Domain four is planning and management. This domain focuses on determining and monitoring work plans, timelines, resources, and other components needed to complete and deliver an evaluation study. Planning and management include networking, developing proposals, contracting, determining work assignments, monitoring progress, and fostering use of evaluation findings (AEA, 2018).
The final domain is interpersonal. This domain focuses on human relations and social interactions that ground evaluator effectiveness for professional practice throughout the evaluation (AEA, 2018). Interpersonal skills include cultural competence, communication, facilitation, and conflict resolution. Faculty mentors for EVAL students create competencybased behavioral anchors as authentication tools to measure learning outcomes based upon observable knowledge, skills, and abilities through the foundational and enrichment experiences (See Table 1).

Vignette
The EVAL Framework can be used to as a guide to frame a student's experience from admission into a program to graduation, enabling them to be prepared for employment in the field of evaluation. Successful implementation of the EVAL Framework includes a combination of traditional courses, enrichment activities and purposeful interactions. For example, Sally applies to the department and expresses an interest in the field of evaluation. Instead of sending the student to visit with faculty about career opportunities and how coursework might support her interests, the program leader pulls out a copy of the EVAL framework and proceeds to explain the importance and relevance of each of the areas and then talks through how topics will be covered through the approaches illustrated. The program leader explains how the scholarly learning communities are organized, how mentors are assigned, and how the department will encourage collaboration activities. The program leader connects Sally to a faculty member who is currently involved in a grant-funded project and encourages Sally to volunteer to work with the faculty member on the evaluation aspect of the project. Sally's engagement in the project is specifically focused on her developing competencies within the AEA domains. The program leader provides a copy of the Outcomes and Competency Domains associated with evaluation so that Sally can readily see how the courses she takes as a student connect with her experience on the project. Articulating the EVAL Framework's evaluation, values, active and experiential learning, and leadership elements to Sally at the beginning of her program allows her to make connections across the concepts, see the importance of the activities and opportunities for engagement, and provides a foundation for critical competencies to be obtained.

EVAL Framework Impact
EVAL can increase the number of competent evaluators for food, agriculture, natural resources, human sciences, and public health programs. EVAL students participate in mentoring, individualized development plans, scholarly learning communities, collaboration, transferable skills, and career guidance to increase the institutional completion rate. Our team utilized students individualized development plans as a point of analysis to understand their progression plan to graduation. EVAL students have access to Texas A&M University's Office for Student Assisted Services and Career Center to improve career learning and development. Both offices offer extensive individualized support services for underrepresented students. The authors measure student's technical competence through outcomes on a core content exam, qualifying exam, dissertation defense rubric, and career placement. We collect, analyze, and report scholarly contribution data through EVAL students reflective writing exercises, and coauthorship on refereed publications. Implementation of the EVAL Framework using an experiential approach ensures development of intellectual capital that will benefit the FANH sciences by contributing to the workforce and society. The authors are developers and implementers of United States Department of Agriculture, National Institute of Food and Agriculture, United States Agency for International Development, National Science Foundation, and other grants. EVAL students are provided opportunities to collaborate with these projects to apply their EVAL knowledge. Holistic concepts and analytical tools from a variety of FANH disciplines have been used previously to train graduate students as researchers and evaluators in the food and agricultural sciences.
The authors have relationships with the private sector, K-12 school districts, consulting organizations, and universities and non-governmental organizations across the globe. EVAL students experience, apply, and reflect on leadership, management, and communication experiences through purposefully developed and executed experiences within and across the funded projects. Mentoring between and among the authorship team and students is developed using Kram's (1985) mentoring phase model. This model ensures protege professional and interpersonal skills growth. Each student leads within a project, develop teamwork strategies, manages a project, and works in the field with principal investigators collecting data. Taking the lead on project dissemination efforts, while being mentored, will give students the experience of managing a proposal team while developing professional communication skills. These linkages enable the authorship team to craft meaningful learning experiences for EVAL students and improve the future impact of food and agricultural sciences programs.
Preparing students as evaluation scholars using the EVAL Framework is a novel approach. Simply knowing evaluation theory and practice does not prepare students to evaluate complex, community-based, multidisciplinary projects. Impact evaluation is increasingly important and required for project teams. The incorporation of cultural values, active and experiential learning, with leadership development provides a broader array of abilities for stakeholder and beneficiary impacts at the local level. It is recommended that this model be tested in a variety of contexts to determine transferability and sustainability of projects. EVAL scholars will be able to approach evaluation with the ability to work in teams, make ethical decisions, and empower local opinion leaders to promote community engagement and continuous improvement.