All in a moment: Continuous response measurement method for analyzing dynamic science communication content

Lauri M. Baker; Ashley McLeod-Morin; Anissa Mattox

doi:10.37433/aad.v5i2.357

Authors

Lauri M. Baker University of Florida, USA https://orcid.org/0000-0002-4241-6077
Ashley McLeod-Morin University of Florida, USA https://orcid.org/0000-0002-8649-9783
Anissa Mattox University of Florida, USA https://orcid.org/0000-0001-6359-5820

DOI:

https://doi.org/10.37433/aad.v5i2.357

Keywords:

perception analyzer dials, real time response measurement, dual processing theory

Abstract

Science communication strategy plays a crucial role in effectively conveying scientific information to target audiences. While various models exist for developing communication strategies, many focus on measuring the effectiveness of communication efforts and adjusting these based on audience feedback. However, traditional methods of evaluating communication effectiveness often measure one component at a time and do not consider how people make decisions in real-life situations. This paper proposes the use of continuous response measurement (CRM) as a method to evaluate science communication in agricultural social science research. CRM allows for real-time measurement of how individuals make decisions in response to dynamic communication content. This paper compares different types of CRM, including in-person and virtual dial CRM, and provides resources for researchers interested in implementing this methodology. The paper also discusses various research designs that can be used with CRM, such as experimental designs, survey designs, time-series designs, focus groups, and coding content and behavior. The benefits and limitations of CRM are outlined, highlighting the need for immediate feedback and real-time response in science communication campaigns. In-person CRM is discussed, including the selection of stimuli, response prompts, data collection procedures, and data analysis. Virtual CRM is also examined, highlighting its advantages in terms of flexibility and cost-effectiveness. The paper concludes by discussing data output and analysis methods for CRM data. Overall, this paper serves as a methodological proposal for the use of CRM in agricultural social science research, emphasizing the importance of real-time measurement and response in science communication.

Downloads

Download data is not yet available.

References

Alzuhn, E. (2023). Moment-to-moment cheat sheet: Online research vs in-person research. DIALSMITH. http://www.dialsmith.com/blog/moment-research-best-online-research/

Baggaley, J. (1987). Continual response measurement: Design and validation. Canadian Journal of Educational Communication, 16(3), 217–238. https://doi.org/10.21432/T2VG8M

Baggaley, J. (1986a). Developing a televised health campaign: I. Smoking prevention. Media in Education & Development, 19, 29–43.

Baggaley, J. (1986b). Developing a televised health campaign: II. Skin cancer prevention. Media in Education & Development, 19, 173–176.

Beville, H. M. (1985). Audience ratings: Radio, television, and cable. Lawrence Erlbaum Associates.

Biocca, F., Neuwirth, K., Oshagan, H., Zhongdang, P., & Richards, J. (1987, May). Prime-and-probe methodology: An experimental technique for studying film and television [Paper presentation]. International Communication Association, Montreal, QC, Canada.

Biocca, F., David, P., & West, M. (1994). Continuous response measurement (CRM): A computerized tool for research on the cognitive processing of communication messages. In A. Lang (Ed.), Measuring psychological responses to media messages (pp. 15–65). Lawrence Erlbaum Associates.

Biocca, F., & David, P. (1990a). How camera distance affects the perception of candidates during a presidential debate [Unpublished manuscript]. Center for Research in Journalism and Mass Communication, University of North Carolina at Chapel Hill.

Biocca, F., & David, P. (1990b, May). Micro-shifts in audience opinions: A second-by-second analysis of the first 1988 presidential debate [Paper presentation]. International Communication Association, Dublin, Ireland.

Bolce, L., De Maio, G., & Muzzio, D. (1996). Dial-in democracy: Talk radio and the 1994 election. Political Science Quarterly, 111(3), 457–481. https://doi.org/10.2307/2151971

Boussalis, C., & Coan, T. G. (2021). Facing the electorate: Computational approaches to the study of nonverbal communication and voter impression formation. Political Communication, 38(1-2), 75–97. https://doi.org/10.1080/10584609.2020.1784327

Box, G. E., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015). Time series analysis: forecasting and control (5th ed.). John Wiley & Sons.

Caspi, A., Bogler, R., & Tzuman, O. (2019). “Judging a book by its cover”: The dominance of delivery over content when perceiving charisma. Group & Organization Management, 44(6), 1067–1098. https://doi.org/10.1177/1059601119835982

Chandler, R., Ross, H., & Guillaume, D. (2022). Innovative perception analysis of HIV prevention messaging for black women in college: A proof of concept study. BMC Public Health, 22(1), 1255. https://doi.org/10.1186/s12889-022-13564-4

Critical Appraisal Skills Programme. (2018). CASP Checklists: Critical Appraisal Checklists. CASP. https://casp-uk.net/casp-tools-checklists

Cummins, R. G., Smith, D. W., Callison, C., & Mukhtar, S. (2018). Using continuous response assessment to evaluate the effectiveness of extension education products. The Journal of Extension, 56(3), 26. https://tigerprints.clemson.edu/joe/vol56/iss3/26

D'Ambrosio, L. (2019, December 15). Understanding the adoption of and education about new auto technologies among older adults. ROSAP. https://rosap.ntl.bts.gov/view/dot/43817

De Neys, W., & Pennycook, G. (2019). Logic, fast and slow: Advances in dual process theorizing. Current Directions in Psychological Science, 28(5), 503-509. https://doi.org/10.1177/0963721419855658

DialSmith. (2023). Our technology. https://www.dialsmith.com/technology/

Glass, G. V., Willson, V. L., and Gottman, J. M. (1975). Design and Analysis of Time Series Experiments. Boulder, CO: Colorado Associated University Press.

Grayot, J. D. (2020). Dual process theories in behavioral economics and neuroeconomics: A critical review. Review of Philosophy and Psychology, 11(1), 105–136. https://doi.org/10.1007/s13164-019-00446-9

Hamel, L., Kearney, A., Kirzinger, A., Lopes, L., Muñana, C., & Brodie, M. (2020, September 10). KFF health tracking poll - September 2020: Top issues in 2020 election, the role of misinformation, and views on a potential coronavirus vaccine. KFF. https://www.kff.org/coronavirus-covid-19/report/kff-health-tracking-poll-september-2020/

Hughes, S. R. (2015). Moving the needle: A comparative analysis of message reception during televised presidential debates [Doctoral dissertation, Texas Tech University]. Texas Tech University Libraries. http://hdl.handle.net/2346/66113

Izenson, B. (2016). Your memory stinks: How flawed recall & memory bias pollute market research and what can be done about it. DIALSMITH. https://www.dialsmith.com/blog/flawed-recall-memory-bias-pollute-market-research-can-done/

Izenson, B. (2023). What is Online Dial Testing? DIALSMITH. https://www.dialsmith.com/blog/what-is-online-dial-testing/

Janis, I. L. (1997). Groupthink. In R. P. Vecchio (Ed.), Leadership: Understanding the dynamics of power and influence in organizations (pp. 163–176). University of Notre Dame Press. https://psycnet.apa.org/record/1997-36918-010

Jennings, F. J., Allen, M. W., & Le Vu Phuong, T. (2021). More plastic than fish: Partisan responses to an advocacy video opposing single-use plastics. Environmental Communication, 15(2), 218–234. https://doi.org/10.1080/17524032.2020.1819363

Jeon, Y. A., & Lang, A. (2021). The vicious cycle of stressors, food advertising and overeating: How stigmatizing anti-obesity PSAs, which precede food commercials, influence subsequent implicit and explicit attitudes toward high-and low-calorie food. Media Psychology, 24(5), 606–636. https://www.doi.org/10.1080/15213269.2020.1773854

Juska, J. M. (2021). Integrated marketing communication: Advertising and promotion in a digital world (2nd ed.). Routledge. https://doi.org/10.4324/9780367443382

Kenen, J., & Roubein, R. (2020, March 31). Why America is scared and confused: Even the experts are getting it wrong. POLITICO. https://www.politico.com/news/2020/03/31/experts-coronavirus-cdc-158313

Keppel, G. (1982). Design and analysis: A researcher's handbook. Prentice Hall.

LaGrande, L. E., Meyers, C., Cummins, G. R., & Baker, M. (2021). A moment-to-moment analysis of trust in agricultural messages. Journal of Applied Communications,105(2), 2. https://doi.org/10.4148/1051-0834.2375

Lawson, C., Fischer, L., LaGrande, L., & Opat, K. (2020). Do touch that dial: A guide to continuous response measurement in agricultural communications. Journal of Applied Communications, 104(3), 1–18. https://doi.org/10.4148/1051-0834.2333

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic Inquiry. Sage Publications, Inc.

Mike, M. R., Rampold, S. D., Telg, R. W., & Lindsey, A. B. (2020). Utilizing extension as a resource in disaster response: Florida extension’s communication efforts during the 2017 hurricane season. Journal of Applied Communications, (104)1, 1–13. https://doi.org/10.4148/1051-0834.2308

Maier, J., Maier, M, Maurer, M., Reinemann, C., & Meyer, V. (2009). Real time response measurement in the social sciences: Methodological perspectives and applications. Peter Lang.

Millard, W. (1989). Research using the Millard System (Televac) [Research report]. W. J. Millard.

O’Cathain, A., Murphy, E., & Nicholl, J. (2010). Three techniques for integrating data in mixed methods studies. bmj, 341. https://doi.org/10.1136/bmj.c4587

Opat, K., Magness, H., & Irlbeck, E. (2018). Blue Bell's Facebook posts and responses during the 2015 Listeria crisis: A Case Study. Journal of Applied Communications, (102)4, 1–16. https://doi.org/10.4148/1051-0834.2232

Opat, K., Irlbeck, E., Cummins, R. G., Li, N., & Boren-Alpizar, A. E. (2022). Get the story straight: Comparing narrative and logical-scientific communication to capture Gen Z’s interest in science podcasts. Journal of Radio & Audio Media, 1–19. https://doi.org/10.1080/19376529.2022.2145481

Page, M. J., McKenzie, J. E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuiness, L. A., Stewart, L. A., Thomas, J., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. bmj 372(71). https://www.doi.org/10.1136/bmj.n71

Palmgren, J. (1996). Analysis of longitudinal data. Statistics in Medicine, 15(11), 123–1232. https://doi.org/10.1002/(SICI)1097-0258(19960615)15:11<1231::AID-SIM282>3.0.CO;2-Z

Palmer, M., & Cunningham, R. (1987). Computerizing the collection and coding of communication interaction data: The AIDE system [Unpublished manuscript]. Communication Arts Dept., University of Wisconsin at Madison.

Papakonstantinou, E., Hargrove, J. L., Huang, C., Crawley, C. C., Canolty, N. L. (2002). Assessment of perceptions of nutrition knowledge and disease using a group interactive system: The Perception Analyzer®. Journal of the American Dietetic Association, 102(11), 1663–1668, https://doi.org/10.1016/S0002-8223(02)90354-8

Paull, A. (2023). Online dial testing FAQ. DIALSMITH. https://www.dialsmith.com/blog/online-dial-testing-faq-frequently-asked-questions/

Percy, L. (2023). Strategic integrated marketing communications (4th ed.). Routledge. https://doi.org/10.4324/9781003169635

Philport, J. (1980). The psychology of viewer program evaluation. In J. Anderson (Ed.), Proceedings of the 1980 technical conference on qualitative television ratings (pp. B1–B17). Corporation for Public Broadcasting.

Pollard, M. S., & Davis, L. M. (2021). Decline in trust in the Centers for Disease Control and Prevention during the COVID-19 pandemic. RAND Corporation. https://www.rand.org/pubs/research_reports/RRA308-12.html

Reeves, B. (1984, June). Attention to television: Psychological theories and chronometric measures (pp. 251–279). University of Wisconsin.

Reeves, B., Rothschild, M., & Thorson, E. (1983). Evaluation of the Tell-Back audience response system [Research Report for ABC]. Mass Communication Research Center, University of Wisconsin-Madison.

Rossiter, J. R., & Thornton, J. (2004). Fear‐pattern analysis supports the fear‐drive model for antispeeding road‐safety TV ads. Psychology & Marketing, 21(11), 945–960. https://psycnet.apa.org/doi/10.1002/mar.20042

Saks, J., Compton, J. L., Hopkins, A., & El Damanhoury, K. (2016). Dialed in: Continuous response measures in televised political debates and their effect on viewers. Journal of Broadcasting & Electronic Media, 60(2), 231–247. http://dx.doi.org/10.1080/08838151.2016.1164164

Schwarzkopf, S. (2021). Ten little jurors in the training camp: a genealogy of audience simulation. Journal of Cultural Economy, 14(6), 732–749. https://doi.org/10.1080/17530350.2021.1882535

Sellnow, T. L., Parker, J. S., Sellnow, D. D., Littlefield, R. S., Helsel, E. M., Getchell, M. C., Smith, J. M., & Merrill, S. C. (2017). Improving biosecurity through instructional crisis communication: Lessons learned from the PEDv outbreak. Journal of Applied Communications, 101(4), 1–15. https://doi.org/10.4148/1051-0834.1298

Silverman, D. (2019). Interpreting qualitative data (6th ed.). SAGE Publications.

Steede, G. M., Meyers, C., Li, N., Irlbeck, E., & Gearhart, S. (2018). A sentiment and content analysis of Twitter content regarding the use of antibiotics in livestock. Journal of Applied Communications, 102(4), 1–16. https://doi.org/10.4148/1051-0834.2225

Tarpley, T. G., Fischer, L. M., Steede, G. M., Cummins, R. G., & McCord, A. (2020). How much transparency is too much? A moment-to-moment analysis of viewer comfort in response to animal slaughter videos. Journal of Applied Communications, 104(2), 6. https://doi.org/10.4148/1051-0834.2302

Thorson, E., & Reeves, B. (1985). Effects of over-time measures of viewer liking and activity during programs and commercials on memory for commercials. In R. Lutz (Ed.), Advances in consumer research (pp. 549-553). Association of Consumer Research.

Villarreal, R. (2010). Integrated marketing communication strategy. Wiley International Encyclopedia of Marketing. https://doi.org/10.1002/9781444316568.wiem01027

Waddell, G., & Williamon, A. (2017). Measuring the audience. In Lee, S. (Ed.), Scholarly research for musicians. (pp. 148-155). Routledge.

West, M., & Biocca, F. (1992). "What if your wife were murdered": Audience responses to a verbal gaffe in the 1988 Los Angeles presidential debates [Paper presentation]. The Annual Conference of the American Association for Public Opinion Research, St. Petersburg, FL.

West, M., Biocca, F., & David, P. (1991). "You're no Jack Kennedy": Audience responses to a verbal barb in the 1988 Omaha vice-presidential debates [Paper presentation]. The Annual Conference of the American Association for Public Opinion Research, Phoenix, AZ.

Williams, M., & Moser, T. (2019). The art of coding and thematic exploration in qualitative research. International Management Review, 15(1), 45-55. http://www.imrjournal.org/uploads/1/4/2/8/14286482/imr-v15n1art4.pdf

Wöllner, C., & Auhagen, W. (2008). Perceiving conductors' expressive gestures from different visual perspectives. An exploratory continuous response study. Music Perception, 26(2), 129-143. https://doi.org/10.1525/mp.2008.26.2.129