A Content Analysis of SBAE Preservice Teacher Disposition Assessments

Effective teaching is complex as it encapsulates professional, interpersonal, and intrapersonal knowledge. Professionalism is often tied to dispositions, although ambiguous definitions and cultural sensitivity make it a difficult task. Therefore, this study aimed to establish a comprehensive list of dispositions associated with SBAE teacher preparation. The purpose of this study was to examine the use of dispositions assessments in school-based agricultural education (SBAE) teacher preparation programs in the North-Central Region of the American Association for Agricultural Education by conducting a content analysis. The sample frame included 22 dispositions


Introduction and Problem Statement
Effective teaching is complex as it encapsulates professional, interpersonal, and intrapersonal knowledge (Schulte et al., 2005).Accordingly, teacher preparation programs should foster the self-awareness required to enact the knowledge and skills needed as an educator (Schussler et al., 2010).Dispositions, "the habits of professional action and moral commitments that underlie an educator's performance," (Council of Chief State School Officers, 2013, p. 6; Council for the Accreditation of Educator Preparation [CAEP], n.d.) were introduced by the Interstate New Teacher Assessment and Support Consortium (InTASC) in the 1990s to focus on how teachers use knowledge and skills (Schussler et al., 2010) and are used as an accreditation requirement for teacher preparation programs (CAEP, 2022).
Within school-based agricultural education (SBAE), the importance of dispositions is highlighted in a model for teacher preparation in agricultural education and the standards for SBAE teacher preparation programs (American Association for Agricultural Educators [AAAE], 2017).However, there is only one study examining dispositions within the context of SBAE, finding SBAE teachers hold differing opinions regarding which dispositions are the most important for preservice teachers (Bachman & Thiel, 2021).Yet, there are several challenges related to dispositions in teacher preparation, including approaches to teaching dispositions (Edwards & Edick, 2006) and the assessment of dispositions (Saltis et al., 2021).Professionalism is often tied to dispositions, although ambiguous definitions and cultural sensitivity (Creasy, 2015;Davis, 2016) make it a difficult task to assess.Therefore, this study aimed to establish a comprehensive list of dispositions associated with SBAE teacher preparation.

Theoretical and Conceptual Framework
This study was informed by the concept of dispositions within the context of teacher preparation programs.Dispositions are critical regarding teacher effectiveness and student achievements (Edwards & Edick, 2006;Flowers, 2006;Schulte et al., 2005).Within the CAEP framework, teacher dispositions are grouped with knowledge and skills, with scholars noting dispositions are contingent on knowledge and skills (Borko et al., 2007).
Dispositions highlight teachers' internal beliefs and values, such as care, fairness, and honesty (Flowers, 2006).It is not enough for teachers to have only content and pedagogical knowledge; they also need to be able to nurture relationships with students and colleagues (Saltis et al., 2021).Dispositions influence teachers' actions (Tiilikainien et al., 2019) and are tied to teacher competence (Shavelson, 2013).While dispositions are complex (Edward & Edick, 2006), Tiilikainien et al. (2019) suggested two categories for dispositions: attention-oriented (e.g., personal and contextual factors) and intention-oriented (e.g., curricular factors).
Dispositions relate to career-choice behavior, including preservice SBAE teachers (Bachman & Thiel, 2021;Lent et al., 2002;Rocca & Washburn, 2008).Preservice teachers bring their internal beliefs, specific ideas, values, and dispositions with them as they enter teacher preparation programs (Flowers, 2006;Hammerness et al., 2005;Saultz et al., 2021).In addition, as preservice teachers engage in learning experiences related to becoming an educator, they rely on their existing dispositions (Lent et al., 2002).Due to the importance of dispositions on teacher effectiveness (Eck et al., 2021) and the impact on future career choices, dispositions should be a focus within teacher preparation programs (Vagi et al., 2019).For teacher candidates to increase their professional dispositions, they need to reflect on their unexamined values and beliefs and work with mentors to provide feedback (Saltis et al., 2021).

Purpose
The purpose of this study was to examine the use of dispositions assessments in pre-service agricultural education teacher education programs in the North-Central Region of AAAE.This study provides an understanding of what and how dispositions are being assessed in agricultural education teacher preparation programs to assist teacher educators in cultivating the dispositions of preservice teachers and their programs.

Methods
This study used a content analysis method (Neuendorf, 2017) to identify and examine the use of dispositions in preservice SBAE teacher education programs.Content analysis is "the systematic, objective, quantitative analysis of message characteristics" (Neuendorf, 2017, p. 1).The sample frame for this study included all agriculture teacher preparation programs in the North-Central Region of AAAE.
According to the AAAE (2020) standing rules and the National Teach Ag Campaign (National Association of Agricultural Educators, n.d.), there are 39 colleges and universities within the North-Central Region of AAAE currently training preservice SBAE teachers.Invitation emails were sent to all schools in the sampling frame, with one to three unique contacts made via email between March and June of 2022.Seventeen institutions provided disposition instruments and handbooks or written policies which outlined how the disposition assessments are utilized within the teacher preparation program, which accounted for a 44% response rate (n = 17).There were seven 1862 land-grant institutions, one 1890 land-grant institution, two public Non-Land Grant Agricultural and Renewable Resources Universities, and three private institutions represented in the sample.
The initial coding frame was deductively developed based on current standards for teacher education and previous dispositions research (Neuendorf, 2017).Neuendorf recommends the coding frame should have categories that are "exhaustive and mutually exclusive," to ensure validity and reliability, as well as to avoid ambiguity during the coding process (Neuendorf, 2017, p. 131).The coding frame was comprised of four sections mirroring the research questions: (a) a list of preservice SBAE teacher dispositions; (b) theories, frameworks, standards, or literature that informed the dispositions; (c) who assessed the dispositions; and (d) when the dispositions were assessed during the preservice teacher education program.
Two members of the research team piloted the coding frame by concurrently coding three institutions' documents to calculate intra-coder reliability.Upon completion of the pilot, intraclass correlation coefficient (ICC) estimates and their 95% confidence intervals were calculated using SPSS based on a mean-rating (k = 2), absolute-agreement, two-way mixedeffects model.Excellent reliability was found between the two coders in the pilot.The average measure of ICC was .973,with a 95% confidence interval from .956 to .984(F (65, 65) = 36.996,p < .001).Discussion about the pilot led to further refinement of the coding frame to ensure consistency and mutually exhaustive categories.The main content analysis was conducted by randomly assigning institutions to the two coders and using the updated coding frame.Upon completion of the independent coding, the data were collapsed into one dataset, and total counts and frequencies were calculated for each objective.

Objective 1
The first objective of this study was to identify which dispositions were included in assessments of preservice SBAE teachers.A total of 22 dispositions were included in the coding frame, and all of them appeared in the assessments at least once, resulting in a total of N = 554 dispositions across the documents.Table 1 outlines the 22 dispositions and their subdefinitions, which were also used to conduct the content analysis.• systems-thinking approach to solving problems-sees how the parts of a system fit and work together • uses systematic processes to solve problems (appropriate data, etc.).
• is strategic in the way they think and plan

Objective 2
The second objective aimed to determine the frequency of dispositions included in preservice SBAE teacher assessments.The dispositions which appeared the most in analyzed documents were reliable and responsible (n = 65), reflective (n = 48); commitment to student-learning (n = 44), commitment to the profession (n = 43), and commitment to diversity, equity, and inclusion (n = 40).The least frequent dispositions included genuine and authentic (n = creative (n = 6), maintains a positive attitude (n = 4), and exhibits leadership skills (n = 3).The frequency and percentages of the dispositions can be found in Table 2.

Objective 3
The third objective of this study was to identify what theories, frameworks, standards, or literature informed the dispositions assessments.Of the 17 collected assessments, (n = 8) did not report what standards or literature assisted the development of the dispositions assessment.Of those assessments which included the influences of the development of the assessments, the majority were informed by other standards outside of the coding frame, including the Danielson framework, K-12 disposition assessments, and biblical verses (n = 10).InTASC standards informed four of the assessment tools, three were informed by State Professional Teaching Standards, and one was informed by CAEP standards.See Table 4 for more details regarding the standards that informed the disposition assessments.

Objective 4
Objective four identified when dispositions were assessed during agricultural education preservice teacher training.Overwhelmingly, 88.24% (n = 15) of assessments were completed during student teaching.Assessments were completed 23.53% of the time during teacher education courses, during field experiences, and prior to admission in the program (n = 4).Other times outside of the coding frame were also identified 41.18% of the time (n = 7), including disciplinary referrals and prior to student teaching.Five programs (29.41%) did not list the timing for the assessments in their materials (n = 5).Table 5 includes additional information about when dispositions assessments were completed.

Objective 5
The final objective aimed to determine who conducted disposition assessments during preservice agricultural education teacher education.The majority (58.82%) of assessments were completed by the university supervisor (n = 10) and the cooperating teacher (n = 10).The student was also responsible for assessing themselves nearly half of the time (n = 8, 47.06%).Rarely were course instructors (17.64%) or academic advisors (11.76%) responsible for this task.Some dispositions guidelines indicated program/licensing coordinators as an assessor of dispositions, which were categorized as other for the purpose if this study.Table 6 includes additional information regarding individuals responsible for completing disposition assessments at the preservice level.

Conclusions, Discussion, and Recommendations
This study aimed to examine the use of disposition assessments in SBAE teacher education programs.While dispositions are a required part of accreditation for teacher preparation programs (CAEP, 2022) and literature provides clear linkages to why dispositions are essential for effective teaching (Edward & Edick, 2006;Schussler et al., 2010), there is lack of clarity surrounding the use of dispositions and how they are measured (Saltis et al., 2020).Dispositions are seen as personal inputs that impact the learning experiences of preservice teachers, which impact their self-efficacy and outcome expectations or their belief that they can be successful as agriculture teachers.
This research provided a beginning list of dispositions utilized by 17 teacher preparation programs in the North Central Region of AAAE.Twenty-two dispositions were identified as being listed at least once, with 554 individual dispositions listed across 17 disposition assessments.Fourteen of the 22 (63%) dispositions appeared in at least half of the assessments, and 12 institutions included at least one disposition related to diversity, equity, and inclusion, echoing the recent requirements for equity in CAEP accreditation assessments (CAEP, 2020).Reliable and responsible was the most frequently listed disposition, and committed to student learning, and reflective dispositions were used by all but one institution.
While teacher preparation programs are measuring similar dispositions, there are differences in terminology indicating a lack of consistency, putting the onus on teacher preparation programs to develop on their own (Flowers, 2006;Saltis et al., 2020).The ambiguity of teacher dispositions was also apparent within the finding of what sources informed the development of assessments.The most noted source was the InTASC standards, which is unsurprising, as InTASC introduced dispositions to focus on how teachers use knowledge and skills (Schussler et al., 2010).However, a wide variety of sources were indicated, including state standards, K-12 teacher assessment tools, and other universities' disposition instruments.
The disposition assessments were mainly used during the student teaching experience.Since disposition assessments were mainly used at the end of the teacher preparation programs, it aligns with the finding that university supervisors and cooperating educators were the most common individuals tasked with completing disposition assessments.It is interesting to note that only 47% of programs indicated having preservice students self-assess themselves using disposition assessments since teacher preparation programs are encouraged to foster a selfawareness of skills and dispositions among their teacher candidates (Schussler et al., 2010).Limitations of the study include the interpretations of the meanings of disposition wording and definitions on the assessments.Additionally, due to the nature of the content analysis, we could only assess what was included in the documents provided to us.Therefore, aspects of the disposition assessment process, such as what informed the creation of the assessment, who is assessing preservice teachers, and when dispositions are assessed, may not be entirely representative of reality.
This study contributes to understanding teacher dispositions, an essential facet of teacher education preparation programs.However, we have additional questions relating to research and practice.Further research should be conducted to understand what influences students' dispositions and their effect on their belief in their ability to become an agriculture teacher.What are the best practices for utilizing dispositions within a teacher education program?When and how should disposition assessments be used?
Regarding practice, teacher preparation programs in SBAE and beyond can utilize this initial list to identify important dispositions to focus on or for programs looking to create or revise their list.Additionally, we encourage teacher preparation programs to incorporate dispositions throughout the entire teacher preparation program and help foster preservice teachers' development of appropriate dispositions.With dispositions linked to effective teaching, this area of study requires more exploration.

Table 1
SBAE Preservice Dispositions and Sub-definitions

Table 2
Frequency Counts of Dispositions in University Assessments (N = 554) attitude (17.65%).Table3includes a comprehensive list of the number of assessments which included the 22 dispositions.

Table 3
Most Common Dispositions Included in University Assessments(N = 17)

Table 4
Standards that Informed the Development of Dispositions Instruments (N = 17)Other included Danielson Framework, other universities' disposition instruments, other assessment tools used at the K-12 level, and biblical verses.

Table 5
Times during pre-service training when dispositions are assessed (N = 17) Other included discipline referrals and prior to student teaching.