Notice of Funding Opportunity (NOFO): Assessment of Language and Structure
OSTLTS Generic Data collection Request
OMB No. 0920-0879
Submitted: April 10, 2019
Program Official/Project Officer
Corinne Fukayama
Public Health Analyst
Office of the Associate Director for Policy and Strategy
Program Performance and Evaluation Office
1600 Clifton Rd. NE, Atlanta, GA 30333
(404) 718-5715
Mwx9@cdc.gov
Section B – Data collection Procedures 3
1. Respondent Universe and Sampling Methods 3
2. Procedures for the Collection of Information 4
3. Methods to Maximize Response Rates Deal with Nonresponse 5
4. Test of Procedures or Methods to be Undertaken 6
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 6
The respondent universe for this information collection consists of 755 state, tribal, local, and territorial (STLT) health department program directors who submitted an application to a new FY 2017 or FY 2018 CDC non-research, domestic Notice of Funding Opportunity (NOFO); self-identified as a STLT department in the application; and was listed as the application’s program point of contact.
The 755 is a subset of a total of 2,013 applications submitted in response to 80 non-research, domestic Notices of Funding Opportunity (NOFOs) in FY2017 – FY 2018. These applications came from STLT health departments as well as non-profit organizations, academic institutions, and other domestic entities. Over half of the applications were from STLT health departments. Some applicants (identified as the program point of contact in the application) applied to more than one non-research, domestic NOFO in FY 2017 – FY 2018. After removing multiple applications from the same applicant, keeping only the most recent application, there were 841 unique applicants from STLT health departments in FY 2017 – FY 2018. There was a median of 15 STLT applicants per NOFO, and this was used as a threshold in order to ensure a full range of responses per NOFO. After STLT applicants to NOFOs with less than 15 STLT applicants were removed, there were 755 STLT applicants who had applied to 21 NOFOs in FY 2017 – FY 2018.
Around 10% of applications list a financial or grants management director as the Program Point of Contact; these individuals may not have been as substantially involved in developing the application compared to a program director. If a Program Point of Contact did not have substantial involvement there will be instructions to forward the assessment to the program director who worked on the application (please see Attachment A – Respondent Breakdown).
Web-based data collection instrument (see Attachment B – Web-Based Data Collection Instrument Word Version)
A total of 755 respondents will be invited to participate in the web-based data collection. No sampling procedures will be employed, and everyone in the respondent universe will be asked to participate in the data collection.
Telephone-based semi-structured interviews (see Attachment D – Telephone Interview Guide)
A subset of the respondent universe will be invited to participate in the telephone-based semi-structured interviews. Given available resources, PPEO will use convenience and purposive sampling to select 20 individuals to participate in the interviews. The following criteria will be used to select participants: 1) Those who have already completed the web-based assessment (rationale: these respondents have already demonstrated a willingness to provide feedback and their feedback will provide greater context to data analysis for a specific NOFO), and 2) a mix of STLT health departments (both in terms of geography and health department type, i.e., state, local, tribal, or territorial). The interviews will provide CDC with detailed examples in a NOFO that respondents felt affected the clarity, consistency, and organization of the application instructions and project information. Participation in interviews is voluntary.
Data will be collected via a web-based data collection instrument and telephone-based semi-structured interviews.
Web-based data collection instrument
Data will be collected via a web-based data collection instrument and respondents will be recruited through a notification (see Attachment E—Web-Based Data Collection Invitation Email) to the respondent universe. The notification email will explain:
The purpose of the data collection, and why their participation is important
Instructions for participating
Method to safeguard their responses
That participation is voluntary
The expected time to complete the instrument
Contact information for the project team
The email will provide a link to the web-based data collection instrument and instructions to base their responses on the NOFO that is identified in the email. The email will be customized for each respondent and include the NOFO number and title of the respondent’s most recent FY 2017 – FY 2018 application. To minimize burden on the respondent, a link to a copy of the identified NOFO will also be included in the email.
There is a possibility that the email recipient (the point of contact identified on the STLT application) may not be the best person within the STLT health department to respond to the web-based data collection instrument. About 10% of STLT applications list a financial or grants management director as the Program Point of Contact; these individuals may not have been as substantially involved in developing the application compared to a program director. The email will have instructions to forward the email to the appropriate person if the email recipient was not substantially involved in developing the application.
Respondents will be given four weeks to complete the web-based instrument. At the beginning of Week 1, all 755 respondents will receive the initial notification email (see Attachment E – Web-Based Data Collection Invitation Email). At the beginning of Week 2 a reminder email will be sent to all respondents urging them to complete the web-based instrument (see Attachment F – Web-Based Data Collection Reminder Email 1). Additional reminder emails will be sent to respondents who have not completed the web-based instrument at the beginning of Week 3 and again at the beginning of Week 4 (see Attachment G – Web-Based Data Collection Reminder Emails 2). The email monitoring function in SurveyMonkey will allow for tracking who has responded to the web-based instrument, but is not linked to response data. Each reminder email will review the purpose of the data collection, instructions for participation, and the link to the instrument. Those who do not respond by the end of Week 4 will be considered non-responders.
Data from the web-based instrument will be exported from SurveyMonkey into an Excel spreadsheet file and stored on a secure drive on the CDC network that is only accessible to project members. Data will be reviewed for completion and quantitative analyses will use descriptive statistics to determine frequency distributions. Qualitative and thematic analysis will be performed on the open-ended questions from the web-based instrument. Responses will be cross-tabulated to compare similarities and differences in clarity, consistency, and organization between different NOFOs. Results may be cross-tabulated in other ways to identify response similarities and differences among sub-groups of respondents, such as funded and unfunded applicants.
Telephone-based semi-structured interviews
Data will also be collected via telephone-based semi-structured interviews and respondents will be recruited through a notification (see Attachment H – Telephone Invitation Email) to a sample of the respondent universe. The notification email will explain:
The purpose of the data collection, and why their participation is important
Instructions for participating
Method to safeguard their responses
That participation is voluntary
The expected time to complete the instrument
Contact information for the project team
Beginning in Week 3, PPEO will use convenience and purposive sampling to select 20 individuals who completed the web-based data collection instrument and invite them to participate in the telephone-based semi-structured interviews. If the invited respondent does not respond within 7 days, the email will be resent. If the respondent does not respond within 7 days of the second email or declines to participate, they will be considered non-responders, and a replacement applicant respondent will be identified and contacted. For respondents who schedule an interview, a reminder email (see Attachment I – Telephone Reminder Email) will be sent two days prior to the interview confirming the time of the interview and availability of the respondent.
Each interview will consist of two analysts (one to lead the interview and the second to record) and the respondent. An interview guide (see Attachment D – Telephone Interview Guide) will be used to guide the discussion. Data from the interviews will be stored on a secure drive on the CDC network that is only accessible to project members. Qualitative and thematic analysis will be performed on the semi-structured interviews. Responses will be cross-tabulated to compare similarities and differences in clarity, consistency, and organization between different NOFOs. Results may be cross-tabulated in other ways to identify response similarities and differences among sub-groups of respondents, such as funded and unfunded applicants.
Although participation in the data collection is voluntary, the project team will make every effort to maximize the rate of response. The data collection instruments were designed with particular focus on streamlining questions to allow for skipping questions based on responses to previous questions, thereby minimizing response burden. For both the web-based data collection and interviews, after the initial invitation to participate is sent out, a series of reminder emails will be distributed to encourage participation.
The estimate for burden hours for both data collection instruments is based on pilot tests by 7 public health professionals. Of this group, 4 participated in web-based data collection instrument pilot and all 7 participated in the telephone-based semi-structured interview pilot.
In the pilot test of the web-based instrument, the average time to complete the instrument including time for reviewing instructions, gathering needed information and completing the instrument, was approximately 20 minutes (range: 10 –25). For the purposes of estimating burden hours, the upper limit of this range (i.e., 25 minutes) is used.
In the pilot test of the semi-structured interviews, the average time to complete the instrument including time for reviewing instructions, gathering needed information and completing the instrument, was approximately 55 minutes (range: 50 – 60). For the purposes of estimating burden hours, the upper limit of this range (i.e., 60 minutes) is used.
Corinne Fukayama
Public Health Analyst
Program Performance and Evaluation Office
Office of the Associate Director for Policy and Strategy
1600 Clifton Road NE
Atlanta, GA 30333
(404) 718-5715
Katherine Franchot
ORISE Fellow
Program Performance and Evaluation Office
Office of the Associate Director for Policy and Strategy
1600 Clifton Road NE
Atlanta, GA 30333
Muz7@cdc.gov
Attachment A – Respondent Breakdown
Attachment B – Web-Based Data Collection Instrument Word Version
Attachment D – Telephone Interview Guide
Attachment E – Web-Based Data Collection Invitation Email
Attachment F – Web-Based Data Collection Reminder Email 1
Attachment G – Web-Based Data Collection Reminder Emails 2
Attachment H – Telephone Invitation Email
Attachment I – Telephone Reminder Email
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |