Stakeholder Interviews for the Evaluation of
the World Trade Center Health Program for
Impact Assessment and Strategic Planning for Translational Research
Supporting Statement – Section B
July 30, 2019
Program Official/Project Officer
LCDR Pattama Ulrich, RN, MPH
World
Trade Center Health
Program
DHHS/USPHS/CDC/NIOSH
513-223-0011
pulrich@cdc.gov
Table of Contents
Respondent Universe and Sampling Methods
Procedures for the Collection of Information
Methods to Maximize Response Rates and Deal with Non Response
Tests of Procedures or Methods to be Undertaken
Statistical and Data Collection Consultants
List of Attachments
Att. A Authorizing Legislation
Att. B 60-Day FRN
Att. C Consent Form
Att. D Interview Discussion Guide
Att. E Recruitment and Reminder emails
Att. F RAND IRB Determination
Att. G Brief Demographic Survey
Att. H NIOSH IRB Determination Form
SUPPORTING STATEMENT B
The interviews will be conducted with representatives of three major stakeholder categories of the World Trade Center Health Program (WTCHP): funders, researchers, and research users. Two of the three major stakeholder categories have key sub-categories. Table 1 shows the categories and sub-categories from which we will recruit for participation in interviews along with rationale for selection. RAND will conduct 20 interviews across the stakeholder categories. Three of the 20 participants are Federal employees (NIOSH employees) acting within the scope of their job responsibilities and are thus excluded from the burden estimate. The burden estimate is based on the 17 respondents who are not Federal employees.
Table 1: Stakeholder categories and sub-categories
Category |
Sub-category [approach] |
Rationale |
# representatives* |
Funder |
Those affiliated with the WTCHP at NIOSH |
Provides NIOSH’s perspective on translational research efforts. |
3 |
Researchers |
WTC Health Registry |
Provides a unique perspective on surveillance of members’ health. |
1 |
|
Principal Investigators of WTCHP-supported research |
Provides insights into research priorities and the process of communicating work to research users. |
4 |
Research Users |
Clinicians caring for WTCHP Members |
Provides perspectives on quality of care and service delivery from an on-the-ground experience. |
2 |
|
Leadership from the WTC Clinical Centers of Excellence and Data Center representatives |
Provides insights on how WTCHP-supported research is viewed by health system leadership (who make decisions about clinical care) and leaders of the 3 Data Centers (who make decisions around clinical care and the flow of clinical data). |
3 |
|
WTCHP Responders and Survivors (State/local govt) |
Provides perspectives on how research impacts personal care. |
3 |
|
WTCHP Responders and Survivors (private citizens) |
Stakeholders whose decision making affects the WTCHP as a whole. |
4 |
Total |
|
20 participants |
|
For all stakeholder categories RAND will recruit participants using purposive sampling, a type of non-probability sampling that will generate a non-representative subset of each stakeholder category. The qualitative research itself is not meant to be generalizable to a larger population but is exploratory to uncover and elaborate on key themes related to WTCHP research.
Participants will first be selected through lists both maintained and generated by NIOSH (e.g., principal investigator lists, clinical centers of excellence clinical leadership, member and scientific advisory committees). For stakeholder categories where contact lists may not be readily available (e.g., members), we will contact, through an introduction by NIOSH, selected advocacy organizations including, but not limited to, Voices of 9/11. We will sample for maximum variation within each stakeholder category, seeking to obtain variation on characteristics such as: experience and familiarity with WTCHP research; type or focus of research conducted (principal investigators); degree of contact or participation with WTCHP activities; and health outcomes experienced (members).
RAND will recruit a total of 20 interviewees, balanced across the 3 stakeholder categories (Table 1). We will include 5 participants in the “researcher” stakeholder category, 12 participants in the “research users” category, and 3 participants in the “funder” category. Participants will be asked open-ended questions using a semi-structured interview discussion guide that covers the following topics (see Attachment D. Interview Protocol):
Stakeholder views on key findings from a large systematic review of WTC-related research conducted in a separate part of this evaluation.
Adherence of WTCHP-supported research to key principles of translational research: relevance, transparency, and usefulness of the research.
Examples of use of the research.
Impact of the research on program outcomes as defined by the Research-to-Care logic model.
Opportunities for future directions for the WTCHP, which is funded through 2090.
A single discussion guide with stakeholder-specific prompts will be used for the semi-structured, in-depth discussions with representatives of all stakeholder categories. The interview questions have been constructed to be general and applicable to all stakeholder categories, allowing for within- and across- group comparisons across questions. Participants are expected to provide different perspectives and levels of detail in their answers, which then can be analyzed to make connections among the data.
Depending on the timing of OMB approval, we anticipate conducting interviews shortly after, most likely in the winter of 2020. If this occurs, results will be analyzed in the spring of 2020. Each telephone interview will last approximately 1 hour and will be recorded and transcribed. Interview discussions will be conducted by one RAND staff member using the standardized interview protocol/script (including suggested question probes). A second RAND staff member will be available to take notes as a back-up to the recording and to keep time.
The following are examples of procedures that have proven effective in previous studies and will be used when possible to obtain at least a 90% response rate:
Informing respondents of project purpose and rationale of the evaluation, who will see the evaluation results, and how the results will be used.
Using culturally appropriate data collection instruments and procedures.
Using alternative communication means, such as video-or teleconferencing.
Addressing data security and privacy safeguards with respondents.
Minimizing the time needed for participation in the project.
Informing respondents about the project process and interview duration and setting, so that they know what to expect.
Limiting participation of each respondent to one interview phone call.
Discussing the importance of the evaluation for different stakeholders and how the findings will be put into action.
Giving participants multiple options for scheduling.
In addition, we will employ flexible sampling procedures to maximize responses (Attachment E for recruitment and reminder emails). For all stakeholder groups in Table 1, RAND will recruit participants using purposive sampling, a type of non-probability sampling that will generate a non-representative subset of each stakeholder group. If initial recruitment efforts do not yield the desired participant numbers across the target stakeholder sub-communities, snowball sampling will be used, where those who have agreed to participate are asked to provide names and contact information for other potential recruits.
NIOSH will initiate contact by email with each prospective participant (see Attachment E, Recruitment and Reminder emails). The initial contact will inform the prospective participant that RAND will be contacting them to invite them to participate in a telephone interview and that participation in this project is voluntary. All NIOSH and RAND recruitment emails will describe the purpose of the research, privacy procedures (see Section 10 in Supporting Statement A), and the benefits that prospective participants can expect from involvement.
At the time participation is confirmed or declined, RAND will ask respondents to provide the names of 2-3 additional referrals in their same stakeholder group, to be contacted first by NIOSH and then by RAND, should the need arise to implement snowball sampling during the recruitment process.
After the first email contact is made, up to 2 reminder emails will be sent (followed by an optional phone call to any difficult-to-reach and under-enrolled stakeholder group) (Attachment F). After second reminder emails have been sent to the initial list of participants, two approaches will be used to intensify recruitment: (1) we will contact the additional potential participants that the invitees provided when they confirmed or declined the invitation and, if needed; (2) we will enlist the assistance of NIOSH to provide additional names of potential participants. Each of these recommended individuals will be contacted by email using the same procedures described above, except that the name of the referring person (at NIOSH, or by an initial invitee) will also be given.
All interviews will be conducted via telephone, supplemented with a webinar in order to present visual aids as appropriate, to maximize convenience for stakeholder communities and thus maximize response rates.
RAND conducted a detailed pre-test using cognitive interviewing techniques of the interview protocol with three members of RAND staff pulled from different non-researcher occupation types (e.g., administrative staff, facilities/mail staff, research support staff). RAND explored in these cognitive interviews whether certain sections and questions performed as expected, to identify any questions and wording that were unclear, and estimate the length of time required to complete the protocol. We were able to clarify the intent of certain questions and probes based on their feedback.
The data collection tools and procedures were designed by the RAND Corporation under the leadership of the individuals named below. The same individuals will also be responsible for collecting and analyzing the data.
Thomas Concannon, Project Director
20 Park Plaza, Suite 920
Boston, MA 02116
617.338.2059 x8615
Laura Faherty, Co-Project Director
20 Park Plaza, Suite 920
Boston, MA 02116
617.338.2059 x8693
Ramya Chari, Interview Lead
1200 South Hayes Street
Arlington, VA 22202-5050
703.413.1100 x5216
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Julie Brown |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |