Analyzing and visualizing repeated-measures needs assessment data using the ranked discrepancy model

Lendel Narine; Amy Harder

doi:10.37433/aad.v5i2.321

Authors

Lendel Narine Utah State University, USA https://orcid.org/0000-0001-6962-2770
Amy Harder University of Connecticut, USA https://orcid.org/0000-0002-7042-2028

DOI:

https://doi.org/10.37433/aad.v5i2.321

Keywords:

needs assessment, ranking, discrepancy, ordinal data, analysis, Borich

Abstract

The Ranked Discrepancy Model was introduced in 2021 as an alternative for analyzing Borich-style competency-based needs assessment data which avoided the pitfalls associated with the original methods for analysis. In this article, we sought to expand upon that work by developing and testing a new framework to analyze and visualize repeated-measures needs assessment data using the Ranked Discrepancy Model (RDM). Data for the analyses were taken from statewide community needs assessments conducted in Utah and Florida with paid survey panelists recruited by an online survey vendor. We found it was possible to apply the RDM to repeated-measures data using Microsoft Excel. A comparison of results obtained from analyzing data using paired t-tests and the RDM model showed strong positive correlations. Additionally, the transition to a spreadsheet format enabled the expansion of data analysis possibilities to include sorting needs by demographic subgroups. We recommend researchers use Excel for the RDM so they can easily examine subgroup needs and apply data visualization techniques to improve the utility of needs assessments and the decisions made by the individuals who interpret the results.

Downloads

Download data is not yet available.

References

Altschuld, J. W., & White, J. L. (2010). Needs assessment: Analysis and prioritization. SAGE. https://doi.org/10.4135/9781452230542

Beck, F., Burch, M., Diehl, S., & Weiskopf, D. (2017). A taxonomy and survey of dynamic graph visualization. Computer Graphics Forum, 36(1), 133-159. https://doi.org/10.1111/cgf.12791

Borich, G. D. (1980). A needs assessment model for conducting follow-up studies. The Journal of Teacher Education, 31(3), 39-42. https://doi.org/10.1177/002248718003100310

Brinker, G. D. (2002). Using standard scores to control for extreme response style bias. Journal of Applied Sociology, 19(2), 81-99. https://www.jstor.org/stable/43481457

Chaudhary, A., & Warner, L. (2022). Identifying gaps between importance and satisfaction to identify extension clients' needs. EDIS. https://edis.ifas.ufl.edu/publication/WC252

Choi, H. J., & Park, J. H. (2022). Exploring deficiencies in the professional capabilities of novice practitioners to reshape the undergraduate human resource development curriculum in South Korea. Sustainability, 14(19), 12121. http://dx.doi.org/10.3390/su141912121

Corder, G. W., & Foreman, D. I. (2014). Nonparametric statistics: a step-by-step approach (2nd ed.). Wiley.

Ghosh, S. K., Burns, C. B., Prager, D. L., Zhang, L., & Hui, G. (2018). On nonparametric estimation of the latent distribution for ordinal data. Computational Statistics and Data Analysis, 119(2018), 86-98. https://doi.org/10.1016/j.csda.2017.10.001

Harder, A., Craig, D., Israel, G., Benge, M., & Caillouet, O. (2023). Exploring the possibilities of a standardized questionnaire for assessing residents’ needs. The Journal of Extension, 61(2), Article 1. https://doi.org/10.34068/joe.61.02.01

Hoekstra, R., Kiers, H. A., & Johnson, A. (2012). Are assumptions of well-known statistical techniques checked, and why (not)? Frontiers in Psychology, 3(2012), 137. https://doi.org/10.3389/fpsyg.2012.00137

Kampen, J., & Swyngedouw, M. (2000). The ordinal controversy revisited. Quality & Quantity, 34(2000), 87–102. https://doi.org/10.1023/A:1004785723554

Kimpston, R. D., & Stockton, W. S. (1979). Needs assessment: A problem of priorities. Educational Technology, 19(6), 16–21. http://www.jstor.org/stable/44421467

Krzywinski, M., & Altman, N. (2014). Nonparametric tests. Nature Methods, 11(5), 467-468. https://doi.org/10.1038/nmeth.2937

Miot, H. A. (2020). Analysis of ordinal data in clinical and experimental studies. Journal Vascular Brasileiro, 19(2020). https://doi.org/10.1590%2F1677-5449.200185

Nahm, F. S. (2016). Nonparametric statistical tests for the continuous data: The basic concept and the practical use. Korean Journal of Anesthesiology, 69(1), 8–14. https://doi.org/10.4097/kjae.2016.69.1.8

Narine, L. K., Ali, A. D., & Hill, P. A. (2021). Assessing rural and urban community assets and needs to inform Extension program planning. Journal of Human Sciences and Extension, 9(1), 8. https://doi.org/10.54718/YYUC3011

Narine, L., & Harder, A. (2021). Comparing the Borich model with the Ranked Discrepancy Model for competency assessment: A novel approach. Advancements in Agricultural Development, 2(3), 96-111. https://doi.org/10.37433/aad.v2i3.169

Pedersen, A. B., Mikkelsen, E. M., Cronin-Fenton, D., Kristensen, N. R., Pham, T. M., Pedersen, L., & Petersen, I. (2017). Missing data and multiple imputation in clinical epidemiological research. Clinical epidemiology, 9, 157–166. https://doi.org/10.2147/CLEP.S129785

Pigott, T. D. (2001). A review of methods for missing data. Educational Research and Evaluation, 7(4), 353-383. https://doi.org/10.1076/edre.7.4.353.8937

Rusticus, S. A., & Lovato, C. Y. (2014). Impact of sample size and variability on the power and type I error rates of equivalence tests: A simulation study. Practical Assessment, Research, and Evaluation, 19(11). https://doi.org/10.7275/4s9m-4e81

Sadiku, M. N. O., Shadare, A. E., Musa, S. M., & Akujuobi, C. M. (2016). Data visualization. International Journal of Engineering Research and Advanced Technology, 2(12), 11-16. https://ijerat.com/index.php/ijerat/article/view/191

Salgado, C. M., Azevedo, C., Proença, H., & Vieira, S. M. (2016). Missing data. In MIT Critical Data (Ed.), Secondary Analysis of Electronic Health Records. Springer. https://doi.org/10.1007/978-3-319-43742-2_13

Sarikaya, A., Correll, M., Bartram, L., Tory, M., & Fisher, D. (2019). What do we talk about when we talk about dashboards? IEEE Transactions on Visualization and Computer Graphics, 25(1), 682-692. https://doi.org/10.1109/TVCG.2018.2864903

Seitz, P., Strong, R., Hague, S., & Murphrey, T. P. (2022). Evaluating agricultural extension agent’s sustainable cotton land production competencies: Subject matter discrepancies restricting farmers’ information adoption. Land, 11(11), 2075. http://dx.doi.org/10.3390/land11112075

Unwin, A. (2020). Why is data visualization important? What is important in data visualization? Harvard Data Science Review, 2(1). https://doi.org/10.1162/99608f92.8ae4d525

U.S. Census Bureau. (n.d.). QuickFacts: Florida, United States. https://www.census.gov/quickfacts/fact/table/FL,US/PST045222

Warner, R. M. (2013). Applied statistics: From bivariate through multivariate techniques (2nd ed.). Sage.

Witkin, B. R. (1994). Needs assessment since 1981: The state of the practice. American Journal of Evaluation, 15(1), 17-27. https://doi.org/10.1177/109821409401500102

Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs assessments: A practical guide. Sage.

Yang, R. K., Fetch, R. J., McBride, T. M., & Benavente, J. C. (2009). Assessing public opinion directly to keep current with changing community needs. Journal of Extension, 47(3), Article 6. https://tigerprints.clemson.edu/joe/vol47/iss3/6