CitSci-SSAv2

Leveraging the emerging field of disaster citizen science to enhance community resilience to improve disaster response

Supporting Statement – Section A

Submitted: 01/10/2018

Project Manager

Mary Leinhos, PhD, MS

Office of Science and Public Health Practice

Office of Public Health Preparedness and Response (OPHPR)

2877 Brandywine Rd., Mailstop K-72

Atlanta, GA 30341

Telephone: (770) 488-8619 Fax: (770) 488-8688
Email: MLeinhos@cdc.gov

Table of Contents

1. Circumstances Making the Collection of Information Necessary

2. Purpose and Use of the Information Collection

3. Use of Improved Information Technology and Burden Reduction

4. Efforts to Identify Duplication and Use of Similar Information

5. Impact on Small Businesses or Other Small Entities

6. Consequences of Collecting the Information Less Frequently

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

9. Explanation of Any Payment or Gift to Respondents

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

12. Estimates of Annualized Burden Hours and Costs

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

14. Annualized Cost to the Federal Government

15. Explanation for Program Changes or Adjustments

16. Plans for Tabulation and Publication and Project Time Schedule

17. Reason (s) Display of OMB Expiration Date is Inappropriate

18. Exceptions to Certification for Paperwork Reduction Act Submissions

List of Attachments

Att. A Authorizing Legislation

Att. B 60-Day FRN

Att. C Consent Forms

Att. D. Interview Guide

Att. E Survey

Att. F Recruitment and Reminder emails

Att. G RAND IRB Determination

Att. H CDC Human Subjects Determination

SUPPORTING STATEMENT A

Submission for “Leveraging the emerging field of disaster citizen science to enhance community resilience to improve disaster response”

• Goal of the study: To explore the potential of disaster citizen science for increasing community resilience, enhancing participation in preparedness and response activities, and improving preparedness efforts; and to develop educational and instructional tools for communities and health departments to navigate the emerging field of disaster citizen science and promote collaborations.

• Intended use of the resulting data: To inform the development of toolkits/guidance to health departments and community groups so that they can align their efforts to increase the benefits and positive impacts of citizen science activities.

• Methods to be used to collect: Individual and group interviews and a nationally representative survey of local health departments.

• The subpopulation to be studied: Citizen scientists and the organizations that represent them; public health officials; local health departments.

• How data will be analyzed: Interview data will be coded for common themes and survey data will be analyzed using descriptive statistics and regression analysis.

1. Circumstances Making the Collection of Information Necessary

This is a request for a new information collection. The Centers for Disease Control and Prevention (CDC) is requesting a 1-year approval to collect information using individual and group interviews and a local health department (LHD) survey.

Citizen science is defined as research activities (e.g., data collection, analysis, and reporting) performed by members of the general public without any particular training in science. Citizen science is growing in popularity, fueled in part by growing use of smartphones and other personal devices in the population. Although citizen collection and use of data during disasters has increased exponentially in recent years and there is great policy interest in the phenomenon, there has been no robust research to date on the use of, barriers to, and impact of citizen science in disasters.

This is an exploratory study and is the first of its kind to explore the growing phenomenon of disaster citizen science. This study will generate information that can help define the phenomenon of disaster citizen science and will result in nationally representative data that can be used as baseline so that changes in citizen science awareness, barriers, and activities can be tracked over time.

The CDC Office of Public Health Preparedness and Response (OPHPR) requests approval of this study to learn about how the emerging field of disaster citizen science can enhance community resilience. This mixed methods information collection using interviews and a cross-sectional survey aims to: (1) assess the potential role of disaster citizen science, including facilitators and barriers, for increasing community resilience, enhancing participation in preparedness and response activities, and improving preparedness efforts; and (2) provide evidence to inform the development of educational and instructional tools for communities and health departments to navigate the emerging field of disaster citizen science and promote collaborations.

Interviews (Attachment D. Interview Guides) with citizen scientists and interested parties will generate hypotheses and provide rich data on the experiences with citizen science to date across all stakeholders active in this enterprise. The collection of nationally-representative survey data (Attachment E. Survey) will generate findings generalizable to the full population of LHDs in the U.S. Identifying information about interview participants will be stored securely and separately from their responses. Interview responses will be combined and reported together, and any quotations used in reports will not be connected with an individual or organization. The names of survey respondents will not be collected or linked to the data respondents provide.

This information collection on citizen science is in accordance with OPHPR’s mission to safeguard health and save lives by providing a platform for public health preparedness and emergency response. As part of its role, OPHPR is empowered to fund applied research to improve the ability of CDC and its partners, including state and LHDs, emergency management organizations, and health care entities, to effectively prepare for and respond to public health emergencies and disasters. The data collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241), included in this request as Attachment A – Authorizing Legislation.

2. Purpose and Use of Information Collection

This one-time information collection will be implemented in collaboration with a contractor and will target citizen scientists and their partners (e.g., academic professionals who work with citizen scientists on research projects) and LHDs in a position to use citizen science data to inform public health decision-making. The purpose of this new information collection is to explore the new field of disaster citizen science. Insights from this information collection will be used to inform journal manuscripts and the development of guidance and toolkits for LHDs and community groups so that they can align their efforts and strengthen the benefits and positive impacts of citizen science activities.

Citizen science organizations lack information on how to organize their activities for ultimate impact, and LHDs lack approaches to respond to and coordinate with citizen science activities within communities. Insights from this information collection will be used to help LHDs and community groups align their efforts and strengthen the benefits and positive impacts of citizen science activities, navigate potential challenges of citizen science efforts that can undermine preparedness and response, and use citizen science activities and data to increase community resilience. This study addresses important gaps in the literature on disaster citizen science. Data from this study will be used to support coordination of citizen science activities in a manner that supports rather than undermines preparedness and response.

3. Use of Improved Information Technology and Burden Reduction

Interviews (Attachment D): When feasible, the contractor organization will employ electronic technology (e.g. video- or tele-conferencing) for conducting interviews and reducing respondent burden. Particular emphasis will be placed on compliance with the Government Paperwork Elimination Act (GPEA), Public Law 105-277, title XVII. The number of questions posed has been held to the minimum required in order to elicit the necessary information. Interview responses and information will be collected according to a standard interview guide (Attachment D) and entered by the interviewer into an electronic file.

Survey (Attachment E): RAND will field the survey via SelectSurvey, a secure platform for the management of online surveys. Survey administration will include computer generated skip patterns that eliminate unnecessary questions to respondents (Attachment E).

4. Efforts to Identify Duplication and Use of Similar Information

OPHPR’s public health mission involves strengthening the evidence-base for decision-making in the event of a public health emergency. As such, it is not expected that any of the information collected under this package is duplicative or is already in the possession of the federal government or other organizations that study or promote disaster citizen science. A large scale literature review has identified a limited evidence base on the use of and potential for citizen science in disasters, supporting the need for this information collection. Although activities, predictors, and barriers to citizen science more broadly have been described in many reports and peer-reviewed journal articles, very few have addressed citizen science for disasters and identified the unique considerations of citizen science for disasters (e.g., need to rapidly deploy a project in response to an unpredictable event, potential to put volunteers at risk). OPHPR is not aware of any duplicative data collection, and will make all reasonable effort to ensure that the information collection does not overlap with other data collection projects to inform disaster response.

5. Impact on Small Businesses or Other Small Entities

Small entities, including small governments (i.e. county and local public health officials), will be included in the proposed information collection. A small government is defined as a government jurisdiction of a city, county, town, township, school district, or special district with a population of less than 50,000. Questions will be held to the absolute minimum required for the intended use when participants could include officials representing small government offices. The maximum time commitment for each participant is 70 minutes for interviews and 30 minutes for surveys.

6. Consequences of Collecting the Information Less Frequently

The proposed one-time information collection is needed in order to learn about an emerging social phenomenon about which little is known, enabling OPHPR to strengthen the evidence base for the potential use of disaster citizen science and promising practices to support its implementation. The lack of information on disaster citizen science may result in efforts by citizen scientist and public health officials that are not aligned and/or mismanaged, which can threaten community resilience and undermine preparedness and response efforts. Burden to individuals participating in the study will be minimized and only necessary questions are included in the collection instruments.

This information collection will inform the development of toolkits and guidance for community groups and health departments so that they can align their efforts in disaster citizen science. This will help to ensure that projects are of higher quality and more likely to actually inform public health decision-making. Findings from this information collection will be used to identify a comprehensive menu of applications of citizen science in disasters, best practices in implementing citizen science projects, and policy barriers to be overcome to encourage greater use of disaster citizen science.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

The 60-day Federal Register notice was published on September 19, 2017, Volume 82, Number 180, Page 43760 (document identifier #CDC-2017-0066). See Attachment B. 60-Day FRN. There were no public comments.

There were no efforts to consult outside the agency.

8b. Consultation

The following agencies and organizations outside of CDC have been consulted on the need for data collection with the audiences, and for the purposes, described in this information collection: the RAND Corporation (the contractor) and the National Association of County and City Health Officials.

9. Explanation of Any Payment or Gift to Respondents

OPHPR will not directly offer cash incentives to the participants targeted in this information collection. However, the contractor organizations will provide small, tangible tokens of appreciation for participants’ time. Each survey participant will receive a $10 gift code and each interview participant will receive $25 gift code after he/she completes the information collection exercise if he/she is allowed to accept it.

The Need for Incentives

Interviews: Providing culturally appropriate incentives is necessary as our target population includes a diverse group (e.g., different types of citizen scientists without professional training in science) that may otherwise be difficult to engage in hour-long interviews. Citizen scientists that engage in citizen-led citizen science efforts (i.e., efforts conducted independently of official data collection typically because citizens are unhappy with the official response) in particular may be distrustful of the government and as such it may be difficult to recruit them to engage in a government-funded research effort. Research suggests that distrust of government and privacy concerns are among the leading reasons why individuals do not participate in government-sponsored data collection efforts.¹ Furthermore, growing fear and distrust are among the reasons why survey response rates have declined across a range of surveys in recent years.²

It is well documented that the government’s responses to recent disasters such as Hurricane Katrina and Hurricane Sandy were considered by many to be lacking, and that the perception of poor government performance leads to greater distrust of government and political institutions.³ We are hoping to interview citizen scientists who experienced disasters and were unsatisfied enough with the official response to take action (and are thus likely to be distrustful of governmental institutions at the present time). Offering a monetary incentive will increase the participation rate of this critical, and otherwise under-represented, group of stakeholders.

Survey: The survey takes approximately 30 minutes to complete and requires an unusual level of mental effort due to long lists of response options and multiple open ended questions. A relatively burdensome survey is necessary for the purposes of this project because this survey is the first of its kind, covering a novel topic area. Most of the questions had to be developed rather than adopted from existing instruments, and as such are new (with respect to content and structure) to survey participants. Long surveys that exceed 20 minutes in length result in declining response rates.⁴ Since the goal of this project is to produce valid and reliable estimates that can be broadly generalized to local health officials across the U.S., minimizing selection bias is an important research goal.

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

OPHPR determined that the Privacy Act is not applicable to this information collection. The contractor will not share PII-linked data with CDC. No information can be retrieved by name of the respondent. Individuals will be responding to the information collection completely voluntarily. Respondents will be informed during the active consent procedures that their responses will be secure and only used by the study team for research purposes (Attachments C, D, E). Information in identifiable form (IIF) will be deleted by the contractor and de-identified data will be provided to OPHPR.

Contractors and the contractor organizations will follow procedures for securing and maintaining privacy during all stages of information collection. Data will be treated in a secure manner and will not be disclosed, unless otherwise compelled by law.

Privacy Impact Assessment Information for Interviews and Survey

1. Participants will be advised of the nature of the information collection activity, the length of time it will require, and that participation is purely voluntary. Participants will be assured that no penalties will occur if they wish not to respond to the information collection as a whole or to any specific questions. These procedures conform to ethical practices for collecting data from human subjects.

2. The proposed information collection has been reviewed and approved by the contractor’s IRB. CDC human subjects approval (Attachment H) has been obtained. Prospective participants will receive information on the purpose and rationale of the project, explanation of what their participation will involve and how their confidentiality will be protected. Prior to the beginning of the information collection, a staff member will address any questions the participants have about the project.

3. All data will be stored in secure electronic files maintained by the contractor and will be accessible only to staff directly involved in the project. All members of the project will be required to sign a statement pledging their personal commitment to guarding the security of collected information. Online information collections will conform completely to federal regulations [the Hawkins-Stafford Amendments of 1988 (P.L. 100-297) and the Computer Security Act of 1987]; all information will be maintained in a password protected secure location. Stored transcripts will not contain personally identifiable information.

4. No system of records is being created for this information collection. No personal identifiable information will be retained.

5. The proposed information collection will not involve collecting or sharing respondents’ personal identification or place of residence with persons outside of the project coordinating organization (the identified contractor). To enable the interview and survey processes, the following Information in Identifiable Form (IIF) will be used: name, phone numbers, email addresses, organizational affiliation and position title. No IIF will be retained.

6. The proposed collection will not impact the respondents’ privacy. All collected information will remain secure. Collected information including interview transcripts and survey data will be entered into appropriate data management systems by contractor staff, and all personal identifying information will be deleted following information verification and cleaning. Final de-identified electronic data (transcripts and summary reports) will be maintained by OPHPR. Analysis and resulting publications will not include any IIF regarding participants.

Interviews: Participants will be recruited from across the U.S. and all participants who are recruited will give consent using Attachment C - Informed Consent form. Recruitment will also be done using the language in the consent form. The contractor will collect and analyze the project specific data. OPHPR will provide technical assistance in the design, implementation, and analysis of the project but will not be in contact with project participants (and will only have access to de-identified data). All information provided by participants will be treated in a secure manner and will not be disclosed unless otherwise compelled by law. Participants will be informed prior to participation that their responses will be treated in a secure manner.

Survey: RAND will use SelectSurvey as the online survey platform, and all participants who are recruited will give consent using Attachment C - Informed Consent form. SelectSurvey application is hosted on a secure RAND server and the website is secured with Secure Sockets Layer (SSL) so that information sent from the client’s computer to the server is encrypted and cannot be interpreted by hackers during transmission. The software, SelectSurvey, is written in .Net. We have the latest release and patches applied which address all known security flaws; the software has a long history of providing a secure environment. The data is stored in a secure Microsoft SQL Server with limited access. RAND's IS Applications group have access to the data on this machine. The machine the survey is on is secured by a number of mechanisms following best practices. These measures address these common issues: 1) Intrusion prevention - on the network and on the host; 2) Monitoring - file system, operating system, and applications; and 3) Patching - operating system and applications.

SelectSurvey.NET Security built into the software:

• All survey ids, report share ids and report ids are encrypted.

• All admin features require login for access

• Different roles for access level permissions are available

• SQL injection prevention scripts are used

• Security built into .NET framework 2.0 or higher is utilized

• Passwords are not stored in the database, they are only used to authenticate to ActiveDirectory when in ActiveDirectory mode

• Different security levels can be set on a per survey basis

• Cross Site Scripting Prevention Code functions to double encode or triple encode text and check for scripting attacks are implemented

• No reusable or identifiable ids are used in the URL query string

• Option to encrypt connection string in database

• Option to use trusted connection for database connection string

• Built in access denied and error pages to handle any security permission issues

• Forms authentication (external instance); ActiveDirectory authentication (internal instance)

After the survey has been fielded, RAND will strip the data of any information that would identify the respondent, per their standard protocols. RAND’s survey protocols and policies conform to participant treatment protocols outlined by the federal Office of Management and Budget, following guidelines from the Belmont Report. Survey responses are secure; personally identifying information is never revealed to clients or other external parties without explicit respondent approval and a client-signed nondisclosure agreement. Surveys are self-administered and accessible any time of day for a designated period. Participants can complete a password-protected survey only once. The survey does not collect personally identifiable data.  Some identifiable information about the respondents is on file with RAND and NACCHO for recruitment purposes and will be deleted once data collection has ended. Researchers conducting the analyses will not obtain any IIF.

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

IRB Approval

The protocols and tools included in this information collection request have been reviewed and approved by the IRB at the contractor organization, and CDC human subjects review has deemed the project to be exempt research. IRB approval letter from the contractor institution and CDC’s human subjects approval notice are included in the attachments (Attachment G. Contractor IRB Determination and Attachment H. CDC IRB Determination).

Justification for Sensitive Questions

Citizen science research typically does not involve questions of a sensitive nature. Some participants may feel uncomfortable answering particular questions about their individual experiences, and/or attitudes. Such questions, when asked, are necessary for the purposes of this information collection. To minimize psychological distress, the moderator and information collection instructions will inform participants that they do not have to respond to any questions they do not want to answer and that they may stop participating at any time. In addition, a subject matter expert from the contractor will be available to answer questions from participants following the information collection activity.

12. Estimates of Annualized Burden Hours and Costs

Interviews (Individual and Group)

This information collection using individual and group interviews will be implemented in collaboration with the contractor and will include citizen scientists and partner organizations (e.g., academic researchers who work with citizen scientists, organizations that provide resources to citizen scientists) and local health officials Information will be collected on uses of disaster citizen science, experience with citizen science, and barriers and facilitators of citizen science as previously described.

We outline the estimated burden hours for the proposed project in Table 12A. The burden table provides estimated annualized burden hours and costs across the different project locations.

A minimum of 35 and a maximum of 55 participants will participate in individual and group interviews led by the contractor. While most individuals will be asked to participate in individual interviews with the study team, in cases where several individuals represent one particular citizen science project or organization, we will allow respondents to participate in a group interview at their request. No single group interview will include more than five participants. We estimate that each interview will last approximately 60 minutes, with an additional 15 minutes required for scheduling (75 minutes total). The maximum total burden for all participants is estimated at 69 hours.

Survey

The survey will be fielded among 600 local health officials as the representative of LHDs; however, we only expect 300 to complete the survey based on previous experiences with this population of respondents. We outline the estimated burden hours for the proposed survey in Table 12C, assuming 300 total respondents. The burden table provides estimated annualized burden hours and costs across the different project locations. We estimate that each survey will require 30 minutes to complete, depending on an individual’s particular route through the survey as a result of skip patterns. The maximum total burden for all participants is estimated at 150 hours.

Table 12-A: Estimated Annualized Burden to Participants for Project

Type of Respondent	Form Name	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Citizen scientists and their partners; local health officials	Interview Guide (semi-structured questionnaire)	55	1	75/60	69
Local health departments	Survey	300	1	30/60	150
TOTAL					219

Table A.12-B presents the calculations for cost of respondents’ time using hourly mean wage information from the U.S. Department of Labor's Bureau of Labor Statistics website, specifically originating from the 2015 National Occupational Employment and Wage Estimates for the United States (http://www.bls.gov/oes/current/oes_nat.htm). The mean wage for Epidemiologists ($37.37) was used to estimate cost burden.

The overall cost of participants’ time for the example information collections is estimated to be a maximum of $8,184.03

The total respondent costs are summarized below in Table A.12-B.

Table A.12-B: Estimated Annualized Cost to Respondents

Type of Respondents	Form Name	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Citizen scientists and their partners; local health officials	Interview Guide	69	$37.37	$2,578.53
Local health departments	Survey	150	$37.37	$5605.50
TOTAL				$8,184.03

*Public wages from http://www.bls.gov/oes/current/oes_nat.htm

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There is no cost or burden to respondents other than their time.

14. Annualized Cost to the Federal Government

There are no equipment or overhead costs. The cost to the federal government includes the salary of CDC staff supporting the data collection activities and associated contractual costs.

Average Annualized Cost of Federal Contract for Information Collection

Contract duration	Total Contract Cost	Contract years covering data collection	Resulting Annualized Contract Cost
Three-year period of performance	$1,238,151	10/12		$343,931

Estimate of Annualized Cost of Federal Employee Involvement

Staff (FTE)	Salary	Fringe (38%)	Total Compensation	% Time	Total Annual
Senior GS-15 Supervising Office Director	$158,503	$60,231	$218,734	2.5	$5,468
Senior GS-14 Project Manager	$124,111	$47,162	$171,273	10	$17,127
Senior GS-14 Project Collaborator	$127,657	$48,510	$176,167	2.5	$4,404
ORISE Fellow GS-11 Equivalent Project Support	$63,161	$24,001	$87,162	5	$4,358
Contracting Officer Rep GS-12	$85,801	$32,604	$118,405	2.5	$2,960
					$34,318

Total Annualized Cost to the Federal Government

Cost Type	Annualized Cost
Federal Contract	$343,931
Federal Employees	$34,318
Total Annualized Cost	$378,249

15. Explanation for Program Changes or Adjustments

This is a new information collection.

16. Plans for Tabulation and Publication and Project Time Schedule

In collaboration with the contractor, OPHPR anticipates starting both the interviews and fielding the survey in Spring 2018 after OMB approval is obtained. OPHPR expects to receive a comprehensive report and interview transcripts from the contractor by September 2018 and preliminary analyses of survey data by December 2018. The contractor will also use these data sources to prepare toolkits and guidance that will be published in Fall 2019, upon completion of the project.

Table A.16.A Project Time Schedule

Activity	Time Schedule
Recruitment emails sent to survey and interview participants	1 month after OMB approval
Information/data collection	1-4 months after OMB approval
Analyses	4-8 months after OMB approval
Publication	18 months after OMB approval

17. Reason(s) Display of OMB Expiration Date is Inappropriate

Display of the OMB expiration date is not inappropriate.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

1 https://www.nap.edu/read/18293/chapter/3#32

2 https://aspe.hhs.gov/system/files/pdf/255531/Decliningresponserates.pdf

3 http://www.stevenpicou.com/pdfs/the-impact-of-hurricane-katrina-on-trust-in-government.pdf; https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5034714/

4 http://www.sciencedirect.com/science/article/pii/S1098301511015245