Development and validation of a high school agricultural literacy assessment

Rose Judd-Murray; Brian K. Warnick; Daniel C. Coster; Max L. Longhurst

doi:10.37433/aad.v5i3.407

Authors

Rose Judd-Murray Utah State University, USA https://orcid.org/0000-0002-2045-1015
Brian K. Warnick Utah State University, USA https://orcid.org/0009-0001-8608-1897
Daniel C. Coster Utah State University, USA https://orcid.org/0000-0002-4801-5637
Max L. Longhurst Utah State University, USA https://orcid.org/0000-0003-3227-800X

DOI:

https://doi.org/10.37433/aad.v5i3.407

Keywords:

National Agricultural Literacy Outcomes, agricultural education, agricultural knowledge, high school, secondary education

Abstract

The National Agricultural Literacy Outcomes (NALOs) are knowledge benchmarks for school-aged youth and are used to improve agricultural literacy (NAITC, 2014; NCAL, 2017). Despite educational efforts, prior research indicated that high school populations remained at low or deficient literacy levels. Additionally, no agricultural literacy assessment instruments using the NALOs as a standardization tool have been developed for the 9-12th grades. The purpose of the study was to validate a summative NALO-centered assessment that could provide baseline data on agricultural literacy following the completion of secondary education (12^th grade). The study followed the framework established by Longhurst et al. (2020) for similar assessments in elementary grades. A Delphi team produced 45 items for validation that were reviewed using a convenience sample of [University] undergraduate students. Those items were evaluated using factor, item, and discriminant analysis. Results finalized two 15-item assessments and determined both had acceptable reliability, were adequate for model fit, and were valid for the NALOs and the three proficiency levels. The instruments are critical tools for providing a standardized approach to evaluation efforts. Researchers and educators should use these instruments to provide comparable agricultural literacy data across populations to better identify trends, program needs, and meaningful inferences.

Downloads

Download data is not yet available.

References

American Association for Agricultural Education (AAAE). (2023). Research values of the American Association for Agricultural Education. https://aaaeonline.org/resources/Documents/FOR%20ONLINE%20(8.5%20x%2011)%20-%20AAAE%20Research%20Values.pdf

Bello, D., Leung, K., Radebaugh, L., Tung, R. L., & van Witteloostuijn, A. (2009). From the editors: Student samples in international business research. Journal of International Business Studies, 40(3), 361–364. https://doi.org/10.1057/jibs.2008.101

Boud, D., & Falchikov, N. (2006). Aligning assessment with long‐term learning. Assessment & Evaluation in Higher Education, 31(4), 399–413. https://doi.org/10.1080/02602930600679050

Brandt, M. R. (2016). Exploring elementary students’ agricultural and scientific knowledge using evidence-centered design [Thesis]. University of Nebraska Lincoln. https://digitalcommons.unl.edu/natresdiss/131/

Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis (2nd ed.). Lawrence Erlbaum.

Cosby, A., Manning, J., Power, D., & Harreveld, B. (2022). New decade, same concerns: A systematic review of agricultural literacy of school students. Education Sciences, 12(4), Article 4. https://doi.org/10.3390/educsci12040235

Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning. Scott, Foresman, and Co.

Doerfert, D. L. (2003). Agricultural literacy: An assessment of research studies published within the agricultural education profession. Proceedings of the 22nd Annual Western Region Agricultural Education Research Conference.

Frick, M. J. (1993). Developing a national framework for a middle school agricultural education curriculum. Journal of Agricultural Education, 34(2), 77–84. https://doi.org/10.5032/jae.1993.02077

Goodman, C. M. (1987). The Delphi technique: A critique. Journal of Advanced Nursing, 12(6), 726–734. https://doi.org/10.1111/j.1365-2648.1987.tb01376.x

Hanel, P. H. P., & Vione, K. C. (2016). Do student samples provide an accurate estimate of the general public? PLoS ONE, 11(12), 1–10. https://doi.org/10.1371/journal.pone.0168354

Hess, A. J., & Trexler, C. J. (2011). A qualitative study of agricultural literacy in urban youth: Understanding for democratic participation in renewing the agri-food system. Journal of Agricultural Education, 52(2), 151–162. https://doi.org/10.5032/jae.2011.02151

Jacobs, J. M. (1996). Essential assessment criteria for physical education teacher education programs: A Delphi study [Doctoral dissertation, West Virginia University]. West Virginia University The Research Repository. https://researchrepository.wvu.edu/cgi/viewcontent.cgi?article=10089&context=etd

Joplin, L. (1981). On defining experiential education. Journal of Experiential Education, 4(1), 17–20. https://doi.org/10.1177/105382598100400104

Judd, R. C. (1972). The use of Delphi methods in higher education. Technological Forecasting and Social Change, 4(2), 173–186. https://doi.org/10.1016/0040-1625(72)90013-3

Lawson, A. E., & Weser, J. (1990). The rejection of nonscientific beliefs about life: Effects of instruction and reasoning skills. Journal of Research in Science Teaching, 27(6), 589–606. https://doi.org/10.1002/tea.3660270608

Leising, J. G., Igo, C. G., Heald, A., Hubert, D., & Yamamoto, J. (1998). A guide to food and fiber systems literacy. W. K. Kellogg Foundation and Oklahoma State University.

Leising, J. G., Pense, S. L., & Igo, C. (2000). An assessment of student agricultural literacy knowledge based on the food and fiber systems literacy framework. Journal of Southern Agricultural Education Research, 50(1), 146–151. http://jsaer.org/category/journal/vol-50/

Longhurst, M. L., Judd-Murray, R., Coster, D. C., & Spielmaker, D. M. (2020). Measuring agricultural literacy: Grade 3-5 instrument development and validation. Journal of Agricultural Education, 61(2), 173–192. https://doi.org/10.5032/jae.2020.02173

MacCallum, R. C., Widaman, K. F., Preacher, K. J., & Hong, S. (2001). Sample size in factor analysis: The role of model error. Multivariate Behavioral Research, 36(4), 611–637. https://doi.org/10.1207/S15327906MBR3604_06

Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741. https://doi.org/10.1037/0003-066X.50.9.741

Minkler, I. M., & Salvatore, A. L. (2012). Study design and analysis in dissemination and implementation research (2nd Edition). Oxford University Press.

National Agriculture in the Classroom. (2014). Agricultural literacy. National Agriculture in the Classroom. https://agclassroom.org/get/literacy/

National Center for Agricultural Literacy. (2017). Multistate research: W3006. Utah State University. https://www.agliteracy.org/research/multistate/

National Research Council. (2009). Transforming agricultural education for a changing world. The National Academies Press. https://doi.org/10.17226/12602

OECD: Programme for International Student Assessment. (2016). PISA 2015: Technical report. Organization for Economic Co-operation and Development (OECD). http://www.oecd.org/pisa/sitedocument/PISA-2015-technical-report-final.pdf

Powell, D., Agnew, D., & Trexler, C. (2008). Agricultural literacy: Clarifying a vision for practical application. Journal of Agricultural Education, 49(1), 85–98. https://doi.org/10.5032/jae.2008.01085

Redmond, E. C., & Griffith, C. J. (2003). Consumer food handling in the home: A review of food safety studies. Journal of Food Protection, 66(1), 130–161. https://doi.org/10.4315/0362-028X-66.1.130

Roberts, T. G. (2006). A philosophical examination of experiential learning theory for agricultural educators. Journal of Agricultural Education, 47(1), 17–29. https://doi.org/10.5032/jae.2006.01017

Sireci, S. G. (1998). The construct of content validity. Social Indicators Research, 45(1), 83–117. https://doi.org/10.1023/A:1006985528729

Specht, A. R., McKim, B. R., & Rutherford, T. (2014). A little learning is dangerous: The influence of agricultural literacy and experience on young people’s perceptions of agricultural imagery. Journal of Applied Communications, 98(3), 63–74. https://doi.org/10.4148/1051-0834.1086

Spielmaker, D. M., & Leising, J. G. (2013). National agricultural literacy outcomes. Utah State University, School of Applied Sciences & Technology. https://cdn.agclassroom.org/nat/data/get/NALObooklet.pdf

Spielmaker, D. M., Pastor, M., & Stewardson, D. M. (2014). A logic model for agricultural literacy programming. Proceedings of the 41st annual meeting of the American Association for Agricultural Education. National Center for Agricultural Literacy. https://www.agliteracy.org/research/logic/

Stevens, C. K. (2011). Questions to consider when selecting student samples. Journal of Supply Chain Management, 47(3), 19–21. https://doi.org/10.1111/j.1745-493X.2011.03233.x

Taber, K. S. (2018). The use of Cronbach's alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273-1296. https://doi.org/10.1007/s11165-016-9602-2

Taylor, R. E., & Judd, L. L. (1989). Delphi method applied to tourism. In S. Witt & L. Moutinho (Eds.), Gazing into the oracle: The Delphi method and its application to social policy and public health (pp. 56–88). Jessica Kingsley Publishers.

Warnick, B. K. (2022). W3006: Multistate agricultural literacy research [2020 Annual Report]. Utah State University. https://www.nimss.org/projects/view/mrp/outline/18611

Winton, B. G., & Sabol, M. A. (2022). A multi-group analysis of convenience samples: Free, cheap, friendly, and fancy sources. International Journal of Social Research Methodology, 25(6), 861–876. https://doi.org/10.1080/13645579.2021.1961187