SupSta A

National Public Health Improvement Initiative: Program Assessment

OSTLTS Generic Information Collection Request

OMB No. 0920-0879

Supporting Statement – Section A

Submitted: June 23, 2014

Program Official/Project Officer

Garry Lowry

Health Scientist

Applied Systems Research and Evaluation Branch

Division of Public Health Performance Improvement

Office of State, Tribal, Local and Territorial Support

Centers for Disease Control

1825 Century Center MS E-70

Phone: 404-498-0361

Email: gel2@cdc.gov

Section A – Justification

1. Circumstances Making the Collection of Information Necessary

Background

This information collection is being conducted using the Generic Information Collection mechanism of the OSTLTS OMB Clearance Center (O2C2) – OMB No. 0920-0879. The respondent universe for this information collection aligns with that of the O2C2. Data will be collected from government staff acting in their official capacities at 21 National Public Health Improvement Initiative (NPHII)-funded State, Tribal, Local, or Territorial (STLT) health agencies. Respondents include 21 Performance Improvement Managers (PIMs) and 42 leaders (i.e., 1 PIM and 2 leaders at each of the 21 selected agencies). This information collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241).

NPHII is a multi-year Cooperative Agreement funded through the Prevention and Public Health Funds. NPHII is designed to strengthen the nation’s health by optimizing agency resource utilization and improving program performance and quality of program services. The program goal is to systematically increase the performance management capacity of public health departments to ensure that public health goals are efficiently and effectively met. The NPHII awardees consist of seventy-three STLT health agencies, including 48 state health departments and the District of Columbia health department; seven federally recognized tribal organizations (three directly funded tribes and four tribal organizations that support approximately 250 federally recognized tribes); nine local health departments that serve large metro populations; four U.S. territories; three U.S.-affiliated Pacific islands; and one Pacific Island organization (Attachment A –NPHII Awardees). In its inaugural year, NPHII awardees were expected to hire a Performance Improvement Manager (PIM) and initiate performance management activities. In subsequent years, NPHII focused more specifically on promoting performance management, quality improvement, and accreditation readiness activities to advance the goal of improving the efficiency and effectiveness of services and programs in STLT awardee organizations.

To assess progress made towards intended outcomes of NPHII, CDC funded the National Network of Public Health Institutes (NNPHI) through a Cooperative Agreement to collaborate with CDC to implement an assessment of NPHII. Past OMB-approved data collections under OMB 0920-0879 included: 1) ‘Annual Assessments of Performance Management and Improvement Practices’ completed by PIMs of the 73 STLTs participating in the NPHII program in years 2 and 3 to capture progress made in the areas of accreditation readiness, performance management and quality improvement; and 2) focus groups with PIMs to help CDC understand how the various programmatic elements of NPHII were or were not supporting awardee achievement of accreditation readiness, performance management and quality improvement.

The results of the ‘Annual Assessments’ provided quantitative data that was used to gauge progress made in these key areas of accreditation readiness, performance management and quality improvement at mid-program (Attachment B – Mid-program report executive summary), and the extent to which their STLT agency’s environment was conducive to achieving these goals. The focus groups provided qualitative data in the form of PIM feedback that was used to strengthen elements of the NPHII program (i.e., recommendations for specific program areas: project officers, PIM network, reporting, technical assistance, and guidance). Results from the ‘Annual Assessments’ and focus groups were also used to assess technical assistance needs and provide program recommendations.

Although the previous collections were used to gauge progress made in key areas (accreditation readiness, performance management, and quality improvement) and to strengthen elements of the NPHII program, the overall impact of NPHII was never assessed. As funding for the NPHII program has been eliminated, there is a need to identify the overall impact of NPHII before the program officially ends on September 29, 2014 and program staff leaves. Assessment questions focused on the impact and perceived value of the initiative have been developed to guide the final phase of the NPHII program assessment (Attachment C –NPHII Summative Assessment Questions and Data Sources).

This information collection request focuses on conducting more in-depth assessments to help CDC understand the impact of CDC or CDC-funded support of NPHII on public health infrastructure. As the initiative ends, it is important to understand the extent to which NPHII supported improved public health practice and performance and its perceived value in improving public health infrastructure. The interviews are intended to complement existing data collection efforts. While other assessment and program data describe areas of success and challenges from the perspective of PIMs, the interviews will help CDC understand the value-added of NPHII from the perspective of other STLT officials (i.e., leadership). This data will allow CDC to better understand the impact of NPHII. Further, the data will provide valuable insight on lessons learned, which may be used to inform current and future performance management and quality improvement initiatives.

One of the objectives of the summative assessment is to understand the value of NPHII and lessons learned to inform current and future performance management and quality improvement initiatives. There is currently no data available to assess which aspects of NPHII are deemed most valuable from various stakeholder perspectives and should be continued or sustained. To help answer this question, CDC has contracted with ICF International to conduct key informant interviews (hereafter referred to as “interviews”) that will provide information on the impact and value of NPHII.

To this end, the purpose of this information collection is to help CDC understand

• the extent to which NPHII supported improved public health practice and performance (i.e., accreditation readiness, quality improvement, and performance management),

• the overall value of NPHII, and

• lessons learned from the four-year initiative.

Privacy Impact Assessment

Overview of the Information Collection System – There are two data collections as part of this assessment: 1) an interviewee request form (Att. D - Interviewee Request Form, Att. E -Screenshots of Interviewee Request Form) and 2) a phone interview (Attachment F –Interview Guide). The interviewee request form and interview guide were each pilot-tested by six CDC NPHII project officers. Feedback from this group was used to refine the questions as needed, and establish the estimated time required to complete the form and interview.

Items of Information to be Collected –

The interviewee request form will ask for contact information for prospective interviewees. Contact information will include names, title/positions, email and phone numbers, and a brief description of the suggested interviewee’s involvement/engagement with NPHII. Contact information for alternative interviewees will also be requested.

The interview guide will have 11 overarching questions:

1. Describe your level of interaction with or engagement in NPHII.

2. Where is NPHII housed within your agency (that is, in what office, department, or center is the initiative managed)? Where is the Performance Improvement Manager (or “PIM”) located, and what is the role of the PIM within your agency?

3. Within the scope of the activities funded by the cooperative agreement, describe the most important accomplishments that have been achieved within your agency with NPHII funding or support.

4. Has NPHII affected how you manage and/or lead within your agency? If so, please describe.

5. Has NPHII changed the overall capacity within your agency to provide high quality programs and services to your customers? If so, please describe.

6. What operational changes, if any, have occurred within your agency due to NPHII?

7. In what ways has NPHII impacted your agency’s ability to work with the public health community and provide service to your community overall?

8. Were there any additional outcomes or changes, intended or not, that occurred as a result of NPHII funding? If so, please describe.

9. What value does having a Performance Improvement Manager (PIM) bring to your agency?

10. What do you perceive as the overall value of NPHII to your agency?

11. Does your agency plan to sustain NPHII-related efforts once CDC NPHII cooperative agreement funding is no longer available? If so, please describe.

Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age

This data collection is not web-based. No website content will be directed at children.

2. Purpose and Use of the Information Collection

The purpose of this data collection is to better understand the overall impact of NPHII on public health infrastructure. The information from this assessment will be combined with other data to provide a comprehensive assessment of NPHII. Specifically, this information collection is to help CDC understand

• the extent to which NPHII supported improved public health practice and performance (i.e., accreditation readiness, quality improvement, and performance management),

• the overall value of NPHII, and

• lessons learned from the four-year initiative.

The comprehensive assessment of NPHII will provide important lessons learned to inform current and future performance management and quality improvement initiatives, as well as an understanding of the overall value of NPHII. The data gathered from this collection will be used to develop a report of the interview results and inclusion of the interview results into summative assessment products (e.g., summary of findings/bullet points, reports). The products will be used, in conjunction with other data on NPHII, to report on the impact of the initiative and to inform current and future performance management, quality improvement, and accreditation readiness initiatives.

Privacy Impact Assessment

Respondents are participating in their official capacity as leaders in state, tribal, local and territorial departments of health. Data collection and use of findings will include respondent name, role and/or title. All identifiable information will be stored in a secure shared drive accessible only to NPHII evaluation team members and select NPHII leadership. Telephone interviews will be recorded with the participant’s verbal permission and transcribed, or detailed notes will be taken by the interviewer. Transcribed documents will be reviewed for accuracy. Once accuracy is ensured, the recordings will be destroyed.

3. Use of Improved Information Technology and Burden Reduction

Data will be collected through a SurveyMonkey interviewee request form and telephone interviews with the support of a telephone interview guide with probes for each question. The interviewee request form was designed in SurveyMonkey to facilitate the ease with which respondents can provide prospective interviewees, and the interview guide was designed to collect the minimum information necessary by an interviewer for the purposes of this project (i.e., limited to 11 questions).

Given the short timeline remaining for the interviews, based on the end of NPHII funding, SurveyMonkey and telephone data collection represent the most expedient way to collect a large amount of data quickly.

4. Efforts to Identify Duplication and Use of Similar Information

There is no information available that can substitute for this data collection as there has been no prior data collection from this respondent universe and for the purposes of assessing the overall value of NPHII. Past NPHII OMB-approved data collections have all been collected from the PIM perspective, and have been conducted through the lens of program feedback. This proposed data collection seeks to obtain the perspective of internal customers (i.e., agency and program leadership), and its purpose is to gauge the impact and value of NPHII rather than for program improvement purposes.

5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this information collection.

6. Consequences of Collecting the Information Less Frequently

This request is for a one time information collection. The purpose of this request is to ensure collection of data that is not otherwise available. The proposed timing is critical since it needs to occur prior to the official end of NPHII, which is September 29, 2014. The consequences of not collecting this information under this mechanism and within these timeframes are as follows:

• Inability to gain perspective of NPHII from a respondent universe other than PIMs, who currently provide all the standardized and programmatic data

• Inability to better understand and articulate how NPHII has contributed to public health practice and performance

• Inability to inform key assessment questions and adequately assess the impact of NPHII

• Inability to assess the overall value of NPHII

• Inability to use data to inform current and future performance management and quality improvement initiatives

There are no legal obstacles to reduce the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances with this information collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

This information collection is being conducted using the Generic Information Collection mechanism of the OSTLTS OMB Clearance Center (O2C2) – OMB No. 0920-0879. A 60-day Federal Register Notice was published in the Federal Register on October 31, 2013, Vol. 78, No. 211; pp. 653 25-26. No comments were received.

CDC partners with professional STLT organizations, such as the Association of State and Territorial Health Officials (ASTHO), the National Association of County and City Health Officials (NACCHO), and the National Association of Local Boards of Health (NALBOH) along with the National Center for Health Statistics (NCHS) to ensure that the collection requests under individual ICs are not in conflict with collections they have or will have in the field within the same timeframe.

9. Explanation of Any Payment or Gift to Respondents

CDC will not provide payments or gifts to respondents.

10. Assurance of Confidentiality Provided to Respondents

The Privacy Act does not apply to this information collection. STLT governmental staff and / or delegates will be speaking from their official roles.

This information collection is not research involving human subjects.

11. Justification for Sensitive Questions

No information will be collected that is of personal or sensitive nature.

12. Estimates of Annualized Burden Hours and Costs

The estimate for burden hours is based on pilot tests of the information collection instruments under conditions that replicated the intended data collection conditions. The interviewee request form and interview guide were each pilot-tested by six CDC NPHII project officers. In the pilot test for the suggested interviewees form, the average time to complete the instrument was approximately 10 minutes, with a range of 5 to 15 minutes. It is estimated that respondents will take approximately 10 minutes to complete the form. In the pilot test for the interview, the average time to complete the instrument including time for reviewing instructions, gathering needed information and completing the instrument, was approximately 40 minutes, with a range of 30 to 44 minutes. Based on these results, the estimated time range for actual respondents to complete the instrument is 45-60 minutes, given feedback from pilot testers that actual interviewees will need more time to respond to questions. It is estimated that the interviews will take 60 minutes.

Estimates for the average hourly wage for respondents are based on the Department of Labor (DOL) National Compensation Survey estimate for management occupations – medical and health services managers in state government (http://www.bls.gov/ncs/ocs/sp/nctb1349.pdf). Based on DOL data, an average hourly wage of $57.11 is estimated for all 42 respondents. Table A-12 shows estimated burden and cost information.

Table A-12: Estimated Annualized Burden Hours and Costs to Respondents

Information collection Instrument: Form Name	Type of Respondent	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Interviewee Request Form (Survey Monkey)	NPHII Performance Improvement Manager	21	1	10/60	3.5	$47.49	$166.22
Interview Guide	Agency or Program Leader at State, Tribal, Local, or Territorial Health Agencies	42	1	1	42	$57.11	$2398.62
	TOTALS	63	1		45.5		$2564.84

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There will be no direct costs to the respondents other than their time to participate in each information collection.

14. Annualized Cost to the Government

Table A-14: Estimated Annualized Cost to the Federal Government

Staff (FTE)	Average Hours per Collection	Average Hourly Rate			Average Cost
Health Scientist (GS14) Assisting with instrument development, OMB package preparation, data quality assurance, data analysis and report preparation	200	$54.87			$10,974.00
Health Scientist (GS14) Assisting with instrument development, OMB package preparation, data quality assurance, data analysis and report preparation	200	$54.87			$10,974.00
External Contractors Instrument development, pilot testing, OMB package preparation, data collection, data coding and entry, quality control, data analysis, and report preparation	-	-			$111,833.00
Estimated Total Cost of Information Collection					$133,781.00

15. Explanation for Program Changes or Adjustments

This is a new information collection.

16. Plans for Tabulation and Publication and Project Time Schedule

Project Time Schedule

NPHII Key Informant Interviews (April 2014 – March 2015)

Develop interview protocol and guides	Completed
Develop sampling strategy	Completed
Pilot test interview guides	Completed
Prepare OMB package	Completed
Submit OMB package	Completed
OMB approval	Tentative 7/2/14
Conduct data collection	Tentative 7/7/14-10/3/14
Collect, code, enter, and quality control of data	Tentative 7/28/14-10/3/14
Analyze data (draft codebook, code interview transcripts, conduct inter-rater reliability checks, refine codebook, synthesize coded data)	10/3/14-2/3/15
Prepare report of data collections results for NPHII summative assessment use	Tentative 1/12/15-2/26/15
Disseminate results/reports for NPHII summative assessment use	Tentative 2/27/15

17. Reason(s) Display of OMB Expiration Date is Inappropriate

We are requesting no exemption.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.

LIST OF ATTACHMENTS – Section A

Att. A – Listing of NPHII Awardees

Att. B – Mid-program report executive summary

Att. C – NPHII Summative Assessment Questions and Data Sources

Att. D – Interviewee Request Form

Att. E - Screenshots of Interviewee Request Form

Att. F – Interview Guide