WTCHP SSA interviews 2019-07-30

Stakeholder Interviews for the Evaluation of

the World Trade Center Health Program for

Impact Assessment and Strategic Planning for Translational Research

Supporting Statement – Section A

July 30, 2019

Program Official/Project Officer

LCDR Pattama Ulrich, RN, MPH

World Trade Center Health Program

DHHS/USPHS/CDC/NIOSH

513-223-0011

pulrich@cdc.gov

Table of Contents

1. Justification

1. Circumstances Making the Collection of Information Necessary

2. Purpose and Use of Information Collection

3. Use of Improved Information Technology and Burden Reduction

4. Efforts to Identify Duplication and Use of Similar Information

5. Impact on Small Businesses or Other Small Entities

6. Consequences of Collecting the Information Less Frequently

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

9. Explanation of Any Payment or Gift to Respondents

10. Protection of the Privacy of Information Provided by Respondents

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

12. Estimates of Annualized Burden Hours and Costs

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

14. Annualized Cost to the Government

15. Explanation for Program Changes or Adjustments

16. Plans for Tabulation and Publication and Project Time Schedule

17. Reason(s) Display of OMB Expiration Date is Inappropriate

18. Exceptions to Certification for Paperwork Reduction Act Submissions

List of Attachments

1. Att. A Authorizing Legislation

2. Att. B 60-Day FRN

3. Att. C Consent Form

4. Att. D Interview Discussion Guide

5. Att. E Recruitment and Reminder emails

6. Att. F RAND IRB Determination

7. Att. G Brief Demographic Survey

8. Att. H NIOSH IRB Determination Form

SUPPORTING STATEMENT A

• Goal of the assessment: NIOSH has contracted with the RAND Corporation to evaluate progress toward translational research funded through the World Trade Center Health Program (WTCHP). We will hold interviews with representatives of different stakeholder groups to explore their perspectives on translational research in the context of the WTCHP.

• Intended use of the resulting data: The results of these interviews will be used to help inform an evaluation of the WTCHP and development of strategic planning recommendations.

• Methods to be used to collect the data: These data will be collected through 20 telephone-based interviews; each interview will last approximately 1 hour.

• The subpopulation to be studied: The interviews will be conducted with WTCHP researchers, research users, and the funder (NIOSH).

• How data will be analyzed: Data from the interviews will be coded for common themes using standard qualitative techniques.

A. Justification

1. Circumstances Making the Collection of Information Necessary

This is a request for a new information collection. The Centers for Disease Control and Prevention (CDC) is requesting a 1-year approval to collect information through semi-structured, in-depth interviews. This data collection is related to the previously approved data collection: “The World Trade Center Health Program: Impact Assessment and Strategic Planning for Translational Research (Part 1, Formative Research: Focus Groups)” (OMB No. 0920-1252, exp. 3/31/2022).

The World Trade Center Health Program (WTCHP) was established by the James Zadroga 9/11 Health and Compensation Act of 2010, Public Law 111-347 (Attachment A- hereafter referred to as “the Zadroga Act”). Under subtitle C, the Zadroga Act requires the establishment of a research program on health conditions resulting from the 9/11 terrorist attacks. The WTCHP currently carries out a robust research program while providing critical monitoring and treatment services for its 75,000 members. Members include responders at the World Trade Center (WTC) and related sites and survivors who were in the New York City disaster area. The program maintains a research mission to identify physical and mental health conditions that may be related to the 9/11 terrorist attacks and define effective diagnostic procedures and treatments for WTC-related health conditions.

In 2016, NIOSH contracted with the RAND Corporation to conduct an independent assessment of the WTCHP Research-to-Care (RTC) model including the research investments to date and the effectiveness with which the Program translates its research to different stakeholder groups. RAND was selected given the project team’s expertise with similar assessments and NIOSH’s requirement for an objective analysis. This work will ultimately provide guidance for the WTCHP on strategic directions, as well as produce findings about the translation of research into improved outcomes for individuals and populations exposed to disasters such as the 9/11 attacks that can be applied elsewhere. As a part of this assessment, we will hold a series of interviews with representatives of different stakeholder groups to explore their perspectives on translational research in the context of the WTCHP. These interviews are necessary to gather information on the translation of WTCHP-supported research into better care for members, the impact of this research, and stakeholders’ views on future directions for the program. Interview responses will be incorporated into RAND’s overall assessment of the WTCHP program’s research portfolio and will inform recommendations for future research investments and strategic direction. They will begin shortly after OMB approval is received.

The RTC model (Figure 1) is the strategic framework employed by the WTCHP to prioritize, conduct, and assess research that informs excellence in clinical care for the population of responders and survivors affected by the 9/11 attack in New York City. The model assumes the collective involvement of WTCHP stakeholders, including members, researchers, clinicians, and program administrators. It accounts for a variety of inputs that can affect the progress and impact of WTCHP research, such as people and organizations, resources, and regulatory rules. The program supports a number of activities that aim to produce tangible outputs such as research findings on WTC-related conditions and healthcare protocols. Finally, the model anticipates short-, intermediate-, and long-term measurement of outcomes and serves as a communication tool for program planning and assessment.

While the RTC model has guided WTCHP research since the Program’s inception, the WTCHP now needs evaluative work to determine whether research investments are resulting in maximal benefits for program stakeholders, and to plan for the program’s strategic priorities. The 2015 reauthorization of the Zadroga Act extended coverage to 2090. As a long-term initiative, periodic assessment of the WTCHP is necessary to ensure a most effective program. The first period of research awarded under the WTCHP has recently come to a close; therefore, an assessment of the effectiveness of the RTC model that has guided WTCHP research to care is timely. This assessment is vital to program improvement and foundational in setting research priorities for continuing highly effective treatment as the program progresses. The ability of the WTCHP to serve its members and realize positive impacts on all of its stakeholders will depend heavily on effective translation of research activities and findings in a manner that is appropriate to diverse stakeholder groups.¹

Figure 1: The WTCHP RTC Model

The present data collection is authorized by Section 51(a) of the Zadroga Act (42 USC Chapter 6A, Subchapter XXXI) (Attachment A).

2. Purpose and Use of Information Collection

The purpose of the interviews is to gather data to help inform an assessment of the WTCHP and development of strategic planning recommendations. In addition, engagement with the WTCHP stakeholders will provide us with insights into how best to communicate with and engage stakeholders at every stage of the RTC process, which is a critical aspect of future strategic planning recommendations.

One main protocol will be used to guide interview sessions across all stakeholder categories (see below for discussion of stakeholder categories). The interview questions have been constructed to be general and applicable to all stakeholder categories, allowing for within- and across-group comparisons across questions (see Attachment D. Interview Protocols).

In addition, a brief (<1 minute) and anonymous demographic survey will be completed by participants at the conclusion of the interview, which will ask for participants’ age, gender, race, ethnicity, role within the WTCHP, and other relevant details about their role (e.g., clinician specialty, WTCHP member type). Collecting this information is important for accurately describing the sample of interview participants, and furthermore, it will provide context for the qualitative data they provide; will lead to more nuanced analysis of their responses; and will illuminate any gaps or imbalances by demographic characteristic.

Specific topics for the interviews include:

• Stakeholder views on key findings from a large systematic review of WTC-related research conducted in a separate part of this assessment.

• Adherence of WTCHP-supported research to key principles of translational research: relevance, transparency, and usefulness of the research.

• Examples of use of the research.

• Impact of the research on program outcomes as defined by the Research-to-Care logic model (Figure 1 above).

• Opportunities for future directions for the WTCHP, which is funded through 2090.

Below we describe the stakeholder groups from which we will recruit, for interview participation, with a goal of balanced participation across the three main stakeholder groups.

• Funders (those affiliated with the WTCHP at NIOSH)

  • Provide NIOSH’s perspective on translational research efforts.

• Researchers

  • WTC Health Registry staff: provide perspectives on surveillance of members’ health.

  • Principal investigators of WTCHP-supported research: provide important insights into research priorities and the process of communicating work to research users.

• Research Users

  • Clinicians from the WTC Clinical Centers of Excellence: provide perspectives on quality of care and service delivery considerations from on-the-ground experience.

  • Leadership from the WTC Clinical Centers of Excellence and Data Center representatives: provide insights on how WTCHP-supported research is viewed by both health system leadership (who make decisions about clinical care) and leaders of the 3 Data Centers (who make decisions around clinical care as well as the flow of clinical data).

  • WTCHP members (responders and survivors): provide member perspectives on how research impacts personal care.

  • Others (policy makers, advisors of the program; non-profits; state/federal agencies that interact with the WTCHP): a group of stakeholders whose decision making affects the WTCHP as a whole.

Each interview will last approximately 1 hour and will be conducted via telephone.

3. Use of Improved Information Technology and Burden Reduction

As noted above, interviews will be held by telephone, supplemented by webinar to show visual aids as appropriate, to minimize burden on the participants.

4. Efforts to Identify Duplication and Use of Similar Information

This is the first assessment of the WTCHP commissioned by NIOSH; therefore this work is novel and not duplicative and to NIOSH’s knowledge, no information collected under this package is already in the possession of the federal government or other organizations involved with the WTCHP. NIOSH will make all reasonable effort to ensure that the information collection does not overlap with other data collection projects related to the WTCHP. While RAND will have collected data through the focus groups noted at the start of this statement (OMB No. 0920-1252, exp. 3/31/2022), these interviews will go much more in depth into certain areas we would like to explore, such as use of the research. In addition, while focus groups are appropriate for generating group debate and discussion, in-depth interviews allow participants to discuss details that they might be uncomfortable discussing in a group setting.

5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection.

6. Consequences of Collecting the Information Less Frequently

The proposed one-time information collection is needed in order to evaluate whether WTCHP research meets translational research principles of being relevant, useful, and transparent for all its stakeholders. Without assessment, WTCHP research investments now and in the future, may not meet intended goals as laid out in the Research-to-Care model with consequences for the health and well-being of Program members. Burden to individuals participating in the assessment will be minimized and only necessary questions are included in the collection instruments.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances associated with this information collection request. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

The 60-day Federal Register notice was published on April 8, 2019 (Vol. 84, No. 67, pp. 13923-13924). No comments were received. No external consultations were conducted.

9. Explanation of Any Payment or Gift to Respondents

There will be no payments or gifts to interviewees.

10. Protection of the Privacy of Information Provided by Respondents

The CDC Privacy Act Officer reviewed this submission and determined that the Privacy Act does not apply.

The brief demographic survey will ask the following questions:

• Year of birth;

• Sex;

• Ethnicity;

• Race;

• WTCHP role in general terms (funder, researcher, research user);

• If a WTCHP member, is the person a general responder, FDNY responder, or survivor

• If a clinician, what is the person’s specialty; and

• Number of years the person has been affiliated with the WTCHP.
  
  

For the telephone interviews, NIOSH and RAND will follow procedures for securing and maintaining privacy during all stages of information collection.

1. Participants will be advised of the nature of the information collection activity (their participation in a telephone interview), the length of time it will require (1 hour including a brief demographic survey that will take less than a minute to fill out), and that participation is purely voluntary. Participants will be assured that no penalties will be incurred if they wish not to respond to the information collection as a whole or to any specific questions. These procedures conform to ethical practices for collecting data from human subjects.

2. Prospective participants will receive the consent form electronically and will provide verbal consent prior to beginning the recorded portion of the telephone interview. Through the consent form, participants will receive information on the purpose and rationale of the project, explanation of what their participation will involve and how their data will be protected (Attachment C). Prior to the beginning of the information collection, a staff member will address any questions the participants have about the project.

3. All data will be stored in secure electronic files maintained by RAND and will be accessible only to staff directly involved in the project. All information will be maintained in a password-protected secure location.

4. The proposed information collection will not involve collecting or sharing respondents’ personal identification or place of residence. No personally identifiable information will be collected (year of birth will be collected but not date); other information on the brief demographic survey is not PII. No IIF will be retained.

5. The proposed collection will not impact the respondents’ privacy. All collected information will remain secure. Collected information includes: interview transcripts which will be entered into appropriate data management systems for qualitative analysis (the researchers will use Dedoose software), with all personal identifying information deleted following information verification and cleaning: and 2) responses to the anonymous brief demographic survey, which will be entered into an Excel spreadsheet that will be password-protected; researchers will have no ability to link individuals who participated in interviews. Final de-identified electronic data (transcripts and summary reports) will be maintained by NIOSH. Analyses will not include any IIF regarding participants.

All interview participants will give consent using an Informed Consent form (Attachment C). Recruitment will also be done using the language in the consent form. RAND will collect and analyze the project specific data. Other than sending initial recruitment emails, NIOSH will not be in contact with project participants (and will only have access to de-identified response data). All information provided by participants will be treated in a secure manner and will not be disclosed unless otherwise compelled by law. Participants will be informed prior to participation that their responses will be treated in a secure manner.

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

Both the RAND IRB (Attachment F) and the WTCHP ADS (Attachment H) have determined that IRB approval is not required.

The 9/11 attacks were a traumatic event that were the impetus for the formation of the WTCHP and are the context for the current project. However, our interview questions focus not on the event itself but rather on stakeholder relations with and perspectives on the WTCHP. The interview questions are not designed to be sensitive in nature, and most of the stakeholders have had a longstanding engagement with the WTCHP so are likely accustomed to thinking about and engaging with their memories of the 9/11 attacks. We acknowledge that for many stakeholders, the WTCHP may be their employer or the funder of their research. To minimize psychological distress, the interviewer and information collection instructions will inform participants that they do not have to respond to any questions they do not want to answer and that they may stop participating at any time.

12. Estimates of Annualized Burden Hours and Costs

We outline the estimated burden hours and respondent costs for the proposed project in Table 12-A. The recruitment strategy for the project is based on three broad categories: WTCHP Funders, WTCHP Researchers, and WTCHP Research Users. Across the three types of interview respondents we are aiming for a total of 20 participants, which will be recruited through purposive sampling to achieve maximum diversity and balance across the 3 main stakeholder categories. Since 3 participants are Federal employees acting within the scope of their job responsibilities (WTCHP Funders), these individuals are excluded from the burden estimate, and the burden table is based on the remaining 17 participants.

The Researcher and Research User (e.g., WTCHP researcher / Center of Excellence leadership or data manager / clinician / member, etc.) include a mix of employees (private sector or state/local government) and private citizens. For purposes of burden estimation, we have further reclassified the functional roles into groups that correspond to categories of Affected Public. We assume that the preponderance of Researchers, leadership from WTC Clinical Centers of Excellence, and Other Stakeholders are based in academic institutions or other units of state and local government. We assume that WTCHP clinicians are predominantly from the private sector, and we assume that WTCHP members (responders and survivors) are either private citizens or current/former employees of state and local government.

Each respondent will participate in a telephone interview that will last approximately 1 hour (see Attachment D). A Brief Demographic Survey (see Attachment G) will be completed at the conclusion of each interview and is included in the 1-hour estimate.

Burden estimates are based on RAND’s prior experience with similar discussion guides and a pilot test conducted in January of 2019 with internal RAND non-research staff. The annual total burden hours are estimated to be 17 hours.

Table 12-A: Estimated Annualized Burden Hours

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)
Principal Investigators of WTCHP-Funded Research	Interview Discussion Guide and Brief Demographic Survey	4	1	1	4
Leadership from WTC Clinical Centers of Excellence	Interview Discussion Guide and Brief Demographic Survey	3	1	1	3
WTC Health Registry staff	Interview Discussion Guide and Brief Demographic Survey	1	1	1	1
Clinicians Caring for WTCHP Members	Interview Discussion Guide and Brief Demographic Survey	2	1	1	2
WTCHP Responders and Survivors (State/local govt)	Interview Discussion Guide and Brief Demographic Survey	3	1	1	3
WTCHP Responders and Survivors (private citizens)	Interview Discussion Guide and Brief Demographic Survey	4	1	1	4
Total		17			17

Table 12-B: Estimated Annualized Burden Cost

The total annualized cost burden is estimated to be a maximum of $906.75, based upon mean hourly wages from the May 2018 Weekly and Hourly Earnings from the New York State Department of Labor.

Type of Respondents	Form Name	Number of Respondents	Average Hourly Wage	Total Burden (in hours)	Total Cost
Principal Investigators of WTCHP-Funded Research	Interview Discussion Guide and Brief Demographic Survey	4	$70.11	4	$280.44
Leadership from WTC Clinical Centers of Excellence and Other Stakeholders;	Interview Discussion Guide and Brief Demographic Survey	3	$80.11	3	$240.33
WTC Health Registry staff	Interview Discussion Guide and Brief Demographic Survey	1	$30.72	1	$30.72
Clinicians Caring for WTCHP Members	Interview Discussion Guide and Brief Demographic Survey	2	$70.11	2	$140.22
WTCHP Responders and Survivors (State/local govt)	Interview Discussion Guide and Brief Demographic Survey	3	$30.72	3	$92.16
WTCHP Responders and Survivors (private citizens)	Interview Discussion Guide and Brief Demographic Survey	4	$30.72	4	$122.88
Total					$906.75

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There is no cost or burden to the participants other than their time to participate in the data collection.

14. Annualized Cost to the Government

The total cost to the Federal Government is $11,197.20.

Staff (FTE)	Average Hours per Collection	Average Hourly Rate	Average Cost
Data collection costs (Contract with RAND)		NA	$3,600.00
Social Science Analyst, GS 15 (Analysis and project management costs)	120	$63.31	$7,597.20
Estimated Total Cost of Information Collection			$11,197.20

15. Explanation for Program Changes or Adjustments

This is a new information collection.

16. Plans for Tabulation and Publication and Project Time Schedule

Depending on the timing of OMB approval, we anticipate conducting interviews shortly after, most likely in the winter of 2020. If this occurs, results will be analyzed in the spring of 2020.

Table 16-A: Project Time Schedule

Activity	Time Schedule
Recruitment emails sent to interview participants	1 week after OMB approval
Information/data collection	1-2 months after OMB approval
Analyses	3-4 months after OMB approval

Standard qualitative techniques will be used for data analysis. Interview transcripts will be entered into qualitative research software and independently coded for common themes by two researchers. To ensure different coders are interpreting the literature as similarly as possible, we will: (1) develop descriptive codebooks that give clear meanings to the use of different codes; (2) perform inter-coder agreement checks prior to analyses where both analysts read the same text, code independently, and discuss areas of disagreement; and (3) perform supervisory reviews at regular time intervals and issue course corrections if necessary. The codebooks will allow for pre-specified themes around the main areas of known interest (e.g., relevance, usability, transparency of WTCHP research), and will also allow for the identification of new themes that arise from the assessment.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

The display of the OMB expiration date is appropriate.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

1 Concannon, Thomas W., et al. "A new taxonomy for stakeholder engagement in patient-centered outcomes research." Journal of General Internal Medicine 27.8 (2012): 985-991.

16