Enhancing effectiveness of Extension program evaluations by validating the trustworthiness of self-reported measures of Extension program outcomes

Arati Joshi; John Diaz; Anil Kumar Chaudhary; K. S. U. Jayaratne; Sebastian Galindo

doi:10.37433/aad.v5i4.519

Authors

Arati Joshi University of Florida, USA https://orcid.org/0000-0002-4946-9366
John Diaz University of Florida, USA https://orcid.org/0000-0002-2787-8759
Anil Kumar Chaudhary Pennsylvania State University, USA https://orcid.org/0000-0001-5809-8854
K. S. U. Jayaratne North Carolina State, USA https://orcid.org/0000-0001-5591-3905
Sebastian Galindo University of Florida, USA https://orcid.org/0000-0002-4061-773X

DOI:

https://doi.org/10.37433/aad.v5i4.519

Keywords:

subjective knowledge, objective knowledge, behavior change, impact assessment, behavioral scaling, SDG 15: Life on Land

Abstract

Assessment of program outcomes in extension often relies on subjective measures, such as perceived or self-reported knowledge, which are criticized for potential bias and inaccuracy. Conversely, objective knowledge, i.e., how much an individual actually knows, is considered more accurate. Studies show varying associations between subjective and objective knowledge, ranging from no correlation to high correlation, and their influence on behavior change also varies. In this study, we aim to quantify the relationship between subjective knowledge, objective knowledge, and behavior change. Data were collected from Master Gardener Volunteer training attendees. We used Pearson correlation and hierarchical linear regressions to explore the relationship between subjective and objective knowledge and their influence on behavior, i.e., engagement in gardening practices. Our findings show that subjective and objective knowledge post-training were moderately correlated, indicating that participants' self-assessments were not entirely accurate before training. Interestingly, only subjective knowledge before training predicted engagement in gardening practices after training, highlighting the significant role of perceived understanding in behavior change. Based on the findings, we suggest that extension programs should focus on addressing participants' existing beliefs to foster enduring behavior change. By designing programs that consider these pre-existing perceptions, extension can more effectively translate knowledge into practical, lasting behaviors.

Downloads

Download data is not yet available.

References

Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T

Ajzen, I., Joyce, N., Sheikh, S., & Cote, N. G. (2011). Knowledge and the prediction of behavior: The role of information accuracy in the theory of planned behavior. Basic and Applied Social Psychology, 33(2), 101–117. https://doi.org/10.1080/01973533.2011.568834

Aqueveque, C. (2018). Ignorant experts and erudite novices: Exploring the Dunning-Kruger effect in wine consumers. Food Quality and Preference, 65, 181–184. https://doi.org/10.1016/j.foodqual.2017.12.007

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295X.84.2.191

Bettinghaus, E. P. (1986). Health promotion and the knowledge-attitude-behavior continuum. Preventive Medicine, 15(5), 475–491. https://doi.org/10.1016/0091-7435(86)90025-3

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). SAGE Publications.

Davis, J. A. (1971). Elementary survey analysis. Prentice-Hall.

Diaz, J. M., Jayaratne, K. S. U., & Kumar Chaudhary, A. (2019). Evaluation competencies and challenges faced by early career extension professionals: Developing a competency model through consensus building. The Journal of Agricultural Education and Extension, 26(2), 1–19. https://doi.org/10.1080/1389224x.2019.1671204

Ellen, P. S. (1994). Do we know what we need to know? Objective and subjective knowledge effects on pro-ecological behaviors. Journal of Business Research, 30(1), 43–52. https://doi.org/10.1016/0148-2963(94)90067-1

Gámbaro, A., Ellis, A. C., & Prieto, V. (2013). Influence of subjective knowledge, objective knowledge and health consciousness on Olive oil consumption—A case study. Food and Nutrition Sciences, 4(04), 445–453. https://doi.org/10.4236/fns.2013.44057

Gonyea, R. M. (2005). Self-reported data in institutional research: Review and recommendations. New Directions for Institutional Research, 2005(127), 73–89. https://doi.org/10.1002/ir.156

Han, T. I. (2019). Objective knowledge, subjective knowledge, and prior experience of organic cotton apparel. Fashion and Textiles, 6(1), 4. https://doi.org/10.1186/s40691-018-0168-7

House, L. O., Lusk, J., Jaeger, S. R., Traill, B., Moore, M., Valli, C., Morrow, B., & Yee, W. (2004, August 1-4). Objective and subjective knowledge: Impacts on consumer demand for genetically modified foods in the United States and the European Union. 2004 American Agricultural Economics Association Annual Meeting, Denver, CO, USA. https://agbioforum.org/wp-content/uploads/2021/02/AgBioForum_7_3_113.pdf

Ienna, M., Rofe, A., Gendi, M., Douglas, H. E., Kelly, M., Hayward, M. W., Callen, A., Klop-Toker, K., Scanlon, R. J., Howell, L. G., & Griffin, A. S. (2022). The relative role of knowledge and empathy in predicting pro-environmental attitudes and behavior. Sustainability, 14(8), 4622. https://doi.org/10.3390/su14084622

Karaca, M., Geraci, L., Kurpad, N., Lithander, M. P., & Balsis, S. (2023). Low-performing students confidently overpredict their grade performance throughout the semester. Journal of Intelligence, 11(10), 188. https://doi.org/10.3390/jintelligence11100188

Kim, M. S., Kim, J., & Thapa, B. (2018). Influence of environmental knowledge on affect, nature affiliation and pro-environmental behaviors among tourists. Sustainability, 10(9), 3109. https://doi.org/10.3390/su10093109

Klerck, D., & Sweeney, J. C. (2007). The effect of knowledge types on consumer‐perceived risk and adoption of genetically modified foods. Psychology & Marketing, 24(2), 171–193. https://doi.org/10.1002/mar.20157

Kluchinski, D. (2014). Evaluation behaviors, skills and needs of cooperative extension agricultural and resource management field faculty and staff in New Jersey. Journal of the NACAA, 7(1). https://www.nacaa.com/journal/c5c991c9-960b-4187-871a-3e027abaccdd

Kotrlik, J. W., Williams, H. A., & Jabor, M. K. (2011). Reporting and interpreting effect size in quantitative agricultural education research. Journal of Agricultural Education, 52(1), 132-142. https://doi.org/10.5032/jae.2011.01132

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121. https://doi.org/10.1037/0022-3514.77.6.1121

Kumar Chaudhary, A., Diaz, J., Jayaratne, K. S. U., & Assan, E. (2020). Evaluation capacity building in the nonformal education context: Challenges and strategies. Evaluation and Program Planning, 79, 101768. https://doi.org/10.1016/j.evalprogplan.2019.101768

Kumar Chaudhary A., & Israel G. D. (2014, December). The savvy survey # 8: Pilot testing and pretesting methods of pretesting (AEC402). Agricultural Education and Communication Department, UF/IFAS Extension. http://edis.ifas.ufl.edu/publication/PD072

Larese-Casanova, M. (2017). Measuring program impacts: 4. Evaluating long-term impacts. Utah State University Extension. https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=2679&context=extension_curall

Liu, L., Liu, Y. P., Wang, J., An, L. W., & Jiao, J. M. (2016). Use of a knowledge-attitude-behaviour education programme for Chinese adults undergoing maintenance haemodialysis: Randomized controlled trial. Journal of International Medical Research, 44(3), 557–568. https://doi.org/10.1177/0300060515604980

Lonka, K., Joram, E., & Bryson, M. (1996). Conceptions of learning and knowledge: Does training make a difference? Contemporary Educational Psychology, 21(3), 240–260. https://doi.org/10.1006/ceps.1996.0021

Macků, K., Caha, J., Pászto, V., & Tuček, P. (2020). Subjective or objective? How objective measures relate to subjective life satisfaction in Europe. ISPRS International Journal of Geo-Information, 9(5), 320. https://doi.org/10.3390/ijgi9050320

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.

O’Leary, J., & Israel, G. (2019). Capturing change: Comparing pretest-posttest and retrospective evaluation methods. University of Florida IFAS Extension, WC135. https://edis.ifas.ufl.edu/publication/WC135

Pieniak, Z., Aertsens, J., & Verbeke, W. (2010). Subjective and objective knowledge as determinants of organic vegetables consumption. Food Quality and Preference, 21(6), 581-588. https://doi.org/10.1016/j.foodqual.2010.03.004

Prochaska, J. O., & Velicer, W. F. (1997). The transtheoretical model of health behavior change. American Journal of Health Promotion, 12(1), 38–48. https://doi.org/10.4278/0890-1171-12.1.38

Redman, A., & Redman, E. (2016). Is subjective knowledge the key to fostering sustainable behavior? Mixed evidence from an education intervention in Mexico. Education Sciences, 7(1), 4. https://doi.org/10.3390/educsci7010004

Rockwell, K., & Bennett, C. (2004). Targeting outcomes of programs: A hierarchy for targeting outcomes and evaluating their achievement. Department of Agricultural Leadership, Education, and Communication: Faculty Publications. http://digitalcommons.unl.edu/aglecfacpub/48/

Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach. SAGE Publications.

Sharma, S. V., Gernand, A. D., & Day, R. S. (2008). Nutrition knowledge predicts eating behavior of all food groups except fruits and vegetables among adults in the Paso del Norte region: Qué Sabrosa Vida. Journal of Nutrition Education and Behavior, 40(6), 361–368. https://doi.org/10.1016/j.jneb.2008.01.004

Waters, E. A., Biddle, C., Kaphingst, K. A., Schofield, E., Kiviniemi, M. T., Orom, H., Li, Y., & Hay, J. L. (2018). Examining the interrelations among objective and subjective health literacy and numeracy and their associations with health knowledge. Journal of General Internal Medicine, 33(11), 1945–1953. https://doi.org/10.1007/s11606-018-4624-2