Credibility in crisis: Determining the availability and credibility of online food supply chain resources during the COVID-19 pandemic

Disruptions from COVID-19 forced agricultural business owners to navigate the uncertainty of market disruptions with limited information. As an effect, the quality of information available for agricultural businesses to adapt to changes was a concern. The purpose of this exploratory study was to determine the availability and credibility of resources for agricultural businesses to make informed decisions about food markets during COVID-19. Source credibility was the guiding framework to achieve the research objectives of 1) Describe resources available related to impacts of COVID-19 on the food supply chain, 2) Determine the credibility of available resources. A quantitative content and textual analyses were employed. Results revealed 401 terms used to describe resources (n = 779). Eleven of the top 36 terms were used over 100 times. These were: farmer, resources, farm, market, business, local, health, safe, supply, agriculture, and chain. The majority of resources (66%, f = 514) were mid-level credible sources (industry/business organization, online/print news source, nonprofit), and 32.2% (f = 251) were of the highest credibility (university scientists, USDA scientist, Extension). Implications of this work show an opportunity for university and Extension systems to publish resources and serve as credible sources related to this particular crisis.


Introduction and Problem Statement
A crisis is a complex event that causes or threatens to cause widespread disaster among a whole social system, often with an urgent timeline forcing rapid responses (Pauchant & Mitroff, 1990). In early 2020, COVID-19 swept across the U.S. creating subsequent crises with system shattering impacts (Sahin et al., 2020), such as obstructive changes in social behaviors (Nakat & Bou-Mitri, 2021). The impacts to agriculture were great with disruptions in food supply chains (Duvall, 2020) and increased need for alternative methods of selling agricultural products to meet new needs (Nelson, 2020). These social behavior changes, specific to agriculture, have continued well into 2021 and are observed as changes in consumers' food buying patterns and disruptions to supply chains (Nakat & Bou-Mitri, 2021).
With the accessibility of traditional avenues and markets being limited or closed, agricultural business owners were left to navigate through the uncertainty and indirect impacts of COVID-19 (Lioutas & Charatsari, 2021). During the pandemic, agriculture and natural resource business owners expressed concern over the quality of information available to help inform decision making (Baker, Kandzer, Rampold, et al., 2020). A stimulus policy report recommended that food and farm business entities should be included in small business support programs and receive the same technical assistance and resources (Thilmany et al., 2020). Extension systems have been positioned as a credible source with adequate online program offerings to serve as a mediator and fill these information gaps in times of crises (Narine & Meier, 2020). However, little is known about what resources currently exist for agricultural business owners related specifically to COVID-19 food supply disruptions. Because of this problem, this study sought to explore and identify available resources and assess the quality of resources, in terms of authoring source credibility. With a better understanding of what topic areas are already covered and the credibility level of those existing resources, researchers and communicators can have insight when working to fill the needs of agricultural business owners navigating COVID-19 impacts.

Theoretical and Conceptual Framework
This study was based on the conceptual framework of source credibility. The premise of source credibility is that the more credible, or trustworthy, the source, the more the receiver of information is to trust and put to use content (Kumkale et al., 2010;Pornpitakpan, 2004;Sternthal et al., 1978). The element of trustworthiness refers to the receiver's perception of the likelihood the information the source is sharing is true (Hovland et al., 1953). There are many characteristics that can be measured or observed to substantiate whether an online source is credible, such as display of credentials of authors, whether the author represents a reputable organization, date of last update, references for the source's data, and source funding (Hawkins, 1999). For example, Burbules (2001) notes specific markers of online credible sources as quality layouts and visually appealing sites, addressing the URL or associated emails for either academic or commercial sources, interpreting whether or not the source takes an authoritative stance or a stance with obvious bias, and if the website has been updated or revised frequently. For the purpose of this study, only one factor related to credibility, authoring source, will be measured. Credibility criteria used in assessing information have been studied within a wide range of disciplines, notably library and information science, marketing, communications, and health sciences (Liu, 2004). According to past source credibility literature, consumers of information tended to perceive authority when information appears "scholarly" (Rieh, 2001). Moreover, perceptions of high authority are found to be associated with academic institutions and government agencies with low authority to commercial sources (Rieh, 2001). Credible sources also link to themselves in the website and include complete reference lists (Burbules, 2001). Although source credibility has not been specifically applied to resources related to mitigating the impacts of COVID-19 on the food supply chain, source credibility has been applied to other areas in agricultural communications. It is known that agricultural communicators should consider the trustworthiness and perceived expertise of sources, such as representatives from regulatory agencies, educational institutions, members of the agriculture sector, or environmental organizations, when developing messages (Lamm et al., 2016). Thus, messages delivered by an individual that is perceived as a credible source may have a more meaningful effect than messages delivered by a source that lacks traits of credibility (Telg et al., 2012). It is suggested that website source credibility often has the same effect.

Purpose
The overall purpose of this study was to explore an area of literature otherwise unknown and to describe resources available to help enable agricultural business owners to make better informed decisions based on credible, trustworthy resources during the COVID-19 pandemic. Thus, helping create alternative methods of buying and selling and lessening the impacts from social behavior changes. The conceptual framework of source credibility guided this study in addressing the following research objectives: RO1) Describe resources available related to the impacts of COVID-19 on the food supply chain. RO2) Determine the credibility, in terms of authoring source, of available resources that aim to help mitigate impacts of food supply chain disruptions from COVID-19.

Methods
This study employed a quantitative content and textual analysis approach using purposive sampling to gather resources in three pre-determined ways. Quantitative content analyses are often used to systematically measure and make sense of materials (Berelson, 1952). The first way the sample was gathered was by using a United States Department of Agriculture (USDA) website that aggregated resources related to the topic of COVID-19 food supply chain disruptions. As of February 2021, this site included 171 resources. Next, the research team developed a list of search terms that could be used to locate resources related to this topic. Each search included the term COVID-19 plus an additional word or phrase, such as farm resources, selling online, buying from farms, local food, food supply chain, regional food, emergency food, curbside pickup, food processing and marketing, food and farm closure, farm legal support, farmworkers, ghost kitchens, food hubs, meatpacking, food boxes, and grocery suppliers. These search terms resulted in 579 novel resources. Resources included in the sample were gathered from the first three pages of the search results, a method based on user search behavior (Law, 2019) and previous work (Baker, McLeod-Morin, Kent, et al., 2020). The third way the sample was collected was through emails with resource links sent to the research team, which resulted in 29 resources. Duplicates were removed in the final stage. The final sample included 779 unique resources.
To describe the sample and assess credibility of resources, researchers developed a codebook, a recommended practice in content analyses (Krippendorff, 2013). This codebook provided guidance to the two coders on how to systematically categorize the resources (variable), in terms of credibility (measurement). The coders conducted a priori coding, where the variables were assessed, and measurement was pre-established based upon literature and framework (Stemler, 2000). The codebook was created by examining characteristics that make a source credible (Hawkins, 1999) and applying these elements to topic areas in this study. For the variable credibility, a scale of 1 (lowest) to 3 (highest) credibility. A resource was coded as a one if there was no clear association/author or it was unknown. Resources were coded as two if the author was a commodity association, co-op, business/industry organization, nonprofit, or a news source. Resources coded as three were considered of the highest credibility and consisted of authors who were university scientists, USDA scientists, university programs, or an Extension system. The codebook was reviewed by a panel of experts for face content and validity, both of which lend credibility to the findings of this study (Krippendorff, 2013). While coding the resources for credibility, the coders also added each resource's introduction or about information to the code sheet. This data was analyzed using a textual analysis software (Anthony, 2021) to describe the resources in the sample by identifying most frequently used terms in the texts.
Two coders, a research assistant and an undergraduate researcher, were trained on how to use and interpret the codebook and were supervised by the lead principal investigator. Prior to coding the full data set, inter-rater reliability was assessed using 20% of the sample and Cohen's Kappa (Cohen, 1960). This test is used to measure the percent of agreement between coders, while also considering coders may agree on measurement by chance (Stemler, 2000). According to Landis and Koch (1977), a Kappa score of .8 is considered substantial and deemed acceptable for social science research. The credibility variable, in this study, was assessed at .8, Cohen's assumptions were satisfied (Cohen, 1960). Three other variables were not considered reliable at the .8 level and were removed from the study. After assessing reliability, the full data set was split between the two coders and coded during the spring of 2021. Disagreements on reliability coding were resolved by random assignment (Lacy et al., 2015), and data were analyzed for descriptive statistics using IBM SPSS 26.

RO1 Description of Available Resources
Researchers identified 779 unique resources during the three-part sampling procedure described in the methods. To understand the content contained within the resources, textual analysis of the resources' introduction and about paragraphs were used. In all, the analysis found 401 individual terms. Singular and plural forms of the same word, as well as differing tenses of the same word were grouped together as one word. In these scenarios, the frequency count of both words were added together. The data analysis software uses a series of three separate dictionaries of words in the English language and excludes overly common words like of, is, the, etc.
When plotted on a line chart, the term frequencies leveled after the top 36 words. Top words that were found to be used over 100 times in resources' introduction and about paragraphs, in descending order, were: farmer, farmers; resource, resources; farm, farms; market, markets; business, businesses; local, locals, locally; health, healthier, healthy; safe, safely, safety; supply, supplies, supplying; agriculture, agricultural; and chain, chains (see Table 1).

RO2 Authoring Source Credibility of Resources
To determine the credibility, in terms of authoring source, of the resources, each resource was coded for its level of credibility based on the codebook. Out of all 779 resources in the sample, two thirds of the resources (f = 514, 66%) were at a mid-level of credibility (2 on a 3-point scale). Example sources from this level would be agricultural commodity associations, agricultural cooperatives, business/industry organizations, nonprofits, and online or print news sources. Approximately one third (f = 251, 32.2%) of all sources were considered to have the highest level of credibility. These sources were university scientists, USDA scientists, universities, university affiliated programs, governmental organizations, or Extension systems. Finally, nearly 2% (f = 14) of all resources exhibited no clear association or author on the resource. Credibility levels can be seen in Table 2.

Conclusions, Discussion, and Recommendations
Limitations specific to this study are that this sample is a snapshot of the resources available at one moment in time. Moreover, this was exploratory research into this specific topic area; therefore, the codebook used was researcher developed, which may include bias in interpretation. This was mitigated through inter-rater reliability but should still be noted. Additionally, some resources may have been missed in the development of the sample.
Results from this study spotlight that there is no shortage of resources available to understand food supply changes and opportunities related to COVID-19 with 779 resources available online related to this topic. Textual analysis from this study showed that general production terms, such as farm(s), farmer(s), and agriculture were highly used among resources. Moreover, terms related to health and safety during COVID-19 were also frequently used, which indicates many resources were following and recommending safe protocols for business operations. Frequency of terms also indicated that of alternative food systems could be being explored by the recurring use of the term local(lly). Topic areas that had less frequently used terms, and therefore less resources available, were restaurants, schools, and deliveries. Though these terms made the top 36 list of terms in the textual analysis, these make up a far less of the percentage of the total frequency for the top words. For entities creating, organizing, or promoting materials to help business owners navigate COVID-19 disruptions to the food supply chain, it is recommended to include resources for less traditional food supply chain partners and processes, such as restaurants, schools, and deliveries. Finally, terms like Extension and research-or science-based were not frequently used. Thus, this could serve as an area of opportunity for Extension systems to publish resources and be considered a credible source in their communities during COVID-19 and future crises involving food supply disruption.
Similarly, in terms of source credibility, the majority of the resources had a mid-level credibility from sources like industry, commodity, or grower organizations, nonprofits, online or print news sources, agricultural cooperatives, etc. This work demonstrates that universities and Extension systems have provided education in this area but not near to the same extent as industry. Part of this is likely due to time constraints and perhaps a focus of quality over quantity, but it does appear there is room for universities and Extension systems to increase available resources on understanding food supply changes and opportunities to sell and buy crops in alternative methods due to COVID-19. Because universities and Extension systems are typically seen as more credible sources, it is possible that people would be more likely to retain and implement information provided by these sources (Kumkale et al., 2010;Pornpitakpan, 2004;Rieh, 2001;Sternthal et al., 1978).
Given the number of resources found for this study, it could be assumed that delving through credible and useful resources, as an agricultural business owner, could be overwhelming. Researchers recommend that university and Extension scientists develop and make available resources on these topics for COVID-19 and future crises, while keeping in mind specific needs or circumstances of different communities. Resources produced from both industry and academia should follow best practices of credible sources, like linking previous related information from themselves, creating visually appealing graphics and layouts on quality websites, providing contact information for websites, and updating information on the sites frequently (Burbules, 2001;Hawkins, 1999). In addition, Extension and academia should look for unique opportunities to disseminate this important information. Possible ideas include collaborating with industry and commodity partners by providing fact sheets and/or infographics that can be shared through traditional media outlets and on social media. Even developing op-ed and blog pieces that can be shared among specific target audiences would be beneficial for getting specific credible information to unique audiences. It is also recommended that no stone be left unturned when marketing and cross-promoting these resources to ensure agricultural business owners see credible resources multiple times, thus adding to the credibility of the source. Since published journal articles can be time restricting, Extension and researchers could develop research-based white papers that can be easily accessed by different organizations and used a source for news stories surrounding the crisis. To aid in future crises and disruptions to the food supply chain, it is recommended that researchers examine information sources from producers, suppliers, business owners, and consumers alike. It would also be wise conduct a needs assessment with agricultural business owners that identifies preferential communication methods as a one component of the research. Both of these efforts could further help in connecting the right resources with the right people. Furthermore, thinking about the bigger picture of future disruptions to the food supply chain, it could be useful to identify opinion leaders among communities and industries to implement and utilize best practices prescribed by credible sources. Lastly, communicators and researchers should work to build relationships with agricultural business owners, whether that is online or in real life, prior to disasters to help audiences easily identify credible sources and seek information there first.
Finally, it is interesting to note that 2% of the resources had no clear association. Although this is a small percentage, it provides a unique opportunity for agricultural communicators to understand the importance of providing sources in their products. This is especially crucial during a crisis when source credibility offers a way to swim through the sea of misinformation.