PHAP ICR- Statement A_Revised 7 15

This information collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241) (See Att. A Public Health Service Act). The Centers for Disease Control and Prevention (CDC) works to protect America from health, safety and security threats, both foreign and in the U.S.¹ CDC strives to fulfill this mission, in part, through a competent and capable public health workforce. One mechanism to developing the public health workforce is through training programs like the Public Health Associate Program (PHAP).

The mission of the Public Health Associate Program (PHAP) is to train and provide experiential learning to early career professionals who contribute to the public health workforce.² PHAP targets recent graduates with bachelors or masters degrees who are beginning a career in public health. Each year, a new cohort of up to 200 associates is enrolled in the program. Associates are CDC employees who complete two-year assignments in a host site (i.e., a state, tribal, local, or territorial health department or non-profit organization). Host sites design their associates' assignments to meet their agency's unique needs while also providing on-the-job experience that prepares associates for future careers in public health. Associates also receive CDC-based training in core public health concepts and topics to provide the knowledge, skills, and abilities necessary to succeed in their assignments and provide a foundation for a career in public health. PHAP hosts an initial in-person orientation and annual public health training at CDC and offers long-distance learning opportunities throughout the program. It is the goal of PHAP that following participation in the two-year program, alumni will seek employment within the public health system (i.e., federal, state, tribal, local, or territorial health agencies, or non-governmental organizations), focusing on public health, population health, or health/healthcare.

When PHAP originated in 2007, the program focused on increasing recruitment and enrollment; to date, there has been limited systematic assessment of the program. As a result, one current program priority is focused on documenting program outcomes to inform refinements to program processes and activities, demonstrate program impact, and inform decision making about future program direction. The purpose of this information collection request (ICR) is to gain a 3 year approval to follow alumni career progression following graduation from PHAP for 380 Alumni. Specifically, the primary respondent pool for the PHAP Alumni Assessment will focus on alumni who graduated in 2014 and beyond. Specifically, PHAP Alumni Group 1 is the Class of 2012 who graduated in 2014. PHAP Alumni Group 2 is the Class of 2013 who will graduate in 2015, and PHAP Alumni Group 3 is the Class of 2014 who will graduate in 2016. There are 100 participants in Group 1-Class of 2012, 135 participants in Group 2-Class of 2013, and 145 participants in Group 3-Class of 2014 who will be followed. Please note, Group 1-Class of 2012 will be given the assessment twice and Groups 2-Class of 2013 and Group 3-Class of 2014 will be given the assessment once, which is explained in detail in statement B. This means the number of estimated responses total to 480, illustrated in the annual burden table in section 12, Table A-12.

The ICR will enable the program to demonstrate evidence of program outcomes, specifically to document how many alumni are retained as members of the public health workforce, where alumni are employed, what topical and functional public health areas alumni support (e.g., chronic disease, infectious disease, assessment, communications, etc.), to what extent alumni support the capabilities of public health agencies at the federal, state, territorial, local, tribal, and non-governmental organizational levels, and to what extent PHAP has influenced alumni career paths (if at all). Information will be used to answer key program assessment questions, specifically: “Is PHAP a quality program?”, “Is PHAP an effective program?”, and “What is the impact of PHAP?”

Overview of the Data Collection System

The information collection system consists of a web-based assessment (see Att. B. PHAP Alumni Assessment) designed to gather information from PHAP alumni following participation in PHAP 1 and 3 years after graduation. This time period was purposefully chosen to allow adequate time to measure alumni career movement and progression. A pilot test of the PHAP Alumni Assessment was conducted with five public health professionals. Feedback from this group was used to refine questions as needed, ensure accurate programming and skip patterns and establish the estimated time required to complete the information collection instrument.

Items of Information to be Collected

Due to the nature of this data collection (i.e. tracking individual alumni careers’ over time), participants cannot be anonymous. However, data collected will remain secure. In order to track individual career progression over time, assessment results will be linked over time using an individualized identification number. However, names will never be stored or linked with assessment results and names will not be used in any reporting or publishing. Assessments will be administered electronically. Each alumnus will receive an embedded link in an email invitation that is unique to that alumnus; each alumnus will only have access to his/her link. The web-based assessment will not require a login. Each alumnus unique survey link will be linked to his or her individualized identification number. Upon clicking on the link, the participant will be taken directly to the beginning of the survey.

The PHAP Alumni Assessment will consist of both closed-ended and open-ended questions. The PHAP Alumni Assessment will collect information about current position status, specifically if alumni are employed, pursuing further education, participating in a training or service program, or pursuing something else. The PHAP Alumni Assessment will contain logic to direct alumni to additional questions that are relevant to their current position status. For example, if an alumnus is currently employed, he/she will be directed to additional questions about his/her employment status. Details surrounding the content of each category of questions (based on position status) are provided below:

Employed: Those who are employed will be asked to provide information on their employer name; employer location; employment setting (e.g., governmental agency, non-profit organization); salary range; if their job focuses on public health or healthcare; public health topical areas they support in their job (e.g., chronic disease, infectious disease); if they support different functional areas of their employer (e.g., monitoring, partner engagement, assessment); position advancement information (e.g., promotion, supervisory status).

Pursuing Further Education: Those who are pursuing further education will be asked to provide information on the primary focus of their education (i.e., public health, health/healthcare, other); university name; university location; area of study; degree sought.

Training or Service Program: Those who are participating in a training or service program will be asked to provide information on their training or service program name and location.

Other: Those who select “other” will be asked to explain what they are pursuing.

All alumni will be asked to provide information on their perceptions of PHAP and how (if at all) PHAP has influenced their career paths.

Assessment questions will remain consistent at each administration (i.e., 1 year or 3 years post-PHAP graduation). The language, however, will be updated for each assessment administration to reflect the appropriate time period. For example, the assessment administered at 1 year post-PHAP will ask alumni to reflect on their experiences within the past year; the assessment administered at 3 years post-PHAP will ask alumni to reflect on their experiences within the past 3 years.

Please note, the purpose, methods, and utility of the approved Fellowship Management System (OMB Control Number 0920-0765) is not consistent with the PHAP alumni information collection request.

The FMS was designed primarily as a program management tool; PHAP uses the FMS to collect proposals from public health agencies and organizations interested in hosting a fellow and to collect potential candidates’ personal statements. This data collection requires a tool better suited for program evaluation purposes. Specifically, this data collection requires an instrument designed using advanced customization and multiple, complex logic / skip patterns and statistical analysis. Therefore, a web-based tool that is better suited to capture robust data for purposes of program evaluation will be used. The platform selected provides the ability to conduct analysis and download data directly into Excel and the preferred statistical software package, SPSS. Given the need for the above, described capabilities, CDC OSTLTS does not use the FMS Directory to collect program evaluation data.

It is important to note that this data collection is confidential and for the purpose of showing the program’s impact; results will only be shared in the aggregate. The purpose is not to connect alumni or share background or employment information with each other. The FMS Directory, used by CDC CSELS for their EIS program, collects Personal Identifiable Information (PII) and is viewable by all of those who input data (e.g., EIS alumni) and can serve as a networking platform; that is not the intention of our data collection.

Therefore, the alumni will not access this survey via the Fellowship Management System (FMS).

Purpose and Use of the Information Collection

The purpose of this information collection request (ICR) is to gain approval to follow alumni career progression following participation in PHAP. The ICR will enable the program to demonstrate evidence of program outcomes, specifically to document how many alumni are retained as members of the public health workforce, where alumni are employed, what topical and functional public health areas alumni support (e.g., chronic disease, infectious disease, assessment, communications, etc.), to what extent alumni support the capabilities of public health agencies at the federal, state, territorial, local, tribal, and non-governmental organizational levels, and to what extent PHAP has influenced alumni career paths (if at all). Information will be used to answer key program assessment questions, specifically: “Is PHAP a quality program?”, “Is PHAP an effective program?”, and “What is the impact of PHAP?” The results of the assessment will be published in peer reviewed journals and/or in non-scientific publications such as practice reports and/or fact sheets.

Use of Improved Information Technology and Burden Reduction

Data will be collected via a web-based assessment allowing respondents to complete and submit their responses electronically. This method was chosen to reduce the overall burden on respondents. The information collection instrument was designed to collect the minimum information necessary for the purposes of this project.

Efforts to Identify Duplication and Use of Similar Information

No prior data collection has been conducted to assess the status of alumni careers, career progression, and/or perceptions of how (if at all), PHAP has influenced alumni career paths.

At its inception, the program was focused on increasing recruitment and enrollment. One current priority of the program is to demonstrate evidence of program impact. This data collection effort will help inform this priority and is a first time attempt.

Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this information collection.

Consequences of Collecting the Information Less Frequently

There are no legal obstacles to reduce the burden. The consequences of not collecting this information would be:

Failure to systematically collect information to document evidence of the effectiveness, value and impact of the PHAP program.
Limited guidance to the program on how to adjust and strengthen the PHAP program.

Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances with this information collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.

Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A 60-day Federal Register Notice (FRN) was published in the Federal Register on 12/23/2014, vol. 79, No. 246, pp. 77006 - 77007. There were no substantial comments from the public. There were no efforts to consult outside the agency.

Explanation of Any Payment or Gift to Respondents

CDC will not provide payments or gifts to respondents.

Assurance of Confidentiality Provided to Respondents

The Privacy Act does apply to this data collection. This data collection is not research involving human subjects.

The PHAP Evaluation team will follow procedures for assuring and maintaining security during all stages of data collection. All information provided by respondents will be treated in a secure manner and will not be disclosed unless otherwise compelled by law. Respondents will be informed prior to participation that their responses will be treated in a secure manner.

Privacy Impact Assessment Information

1. Overview of Data Collection System

The information collection system consists of a web-based assessment (see Att. B. PHAP Alumni Assessment Word Version and Att. C. PHAP Alumni Assessment: Web Version) designed to gather information from PHAP alumni 1 year and 3 years post PHAP graduation, in an effort to allow adequate time to measure career progression.

2. Items of Information to be Collected

The PHAP Alumni Assessment will gather information on the status of alumni careers, career progression, and perceptions of how (if at all) PHAP has influenced alumni career paths. The Assessment contains 52 items and will be administered at two different time points (1 year and 3 years post-graduation) to PHAP alumni. Assessment questions will remain consistent at each administration (i.e., 1 year or 3 years post-PHAP graduation). The language will be updated for each assessment administration to reflect the appropriate time period. For example, the assessment administered at 1 year post-PHAP will ask alumni to reflect on their experiences within the past year; the assessment administered at 3 years post-PHAP will ask alumni to reflect on their experiences within the past 3 years.

The PHAP Alumni Assessment will consist of both closed-ended and open-ended questions.

The PHAP Alumni Assessment will collect information about current position status, specifically if alumni are employed, pursuing further education, participating in a training or service program, or pursuing something else. The Alumni Assessment will contain logic to direct alumni to additional questions that are relevant to their current position status. For example, if an alumnus is currently employed, he/she will be directed to additional questions about his/her employment status.

No personal identifying information will be collected on the PHAP Alumni Assessment. However, the PHAP Alumni Assessment will be distributed to alumni using contact information that is collected during from them during their time as federal employees with PHAP . Prior to graduation, program participants will be asked to provide contact information (i.e., email address) to the PHAP program for how they can be reached after graduation.

Each participant will be assigned a unique identification number so that data collected and maintained from the PHAP Alumni Assessment can be tracked over time, but it will not be retrievable by name. The PHAP Alumni Assessment will be distributed to each participant containing an embedded link that is unique to that participant; each participant will only have access to his/her link.

The spreadsheet linking unique ID numbers to personal identifiable information (i.e., name, email address) will be maintained in a separate, password-protected record file on CDC file servers. Procedures will be followed to limit the linkage of ID numbers and personal identification information.

3. A description of how the information will be shared and for what purpose

All data collected will be shared in the aggregate. The ICR will enable the program to demonstrate evidence of program outcomes, specifically to document how many alumni are retained as members of the public health workforce, to what extent alumni support the capabilities of public health agencies at the federal, state, territorial, local, tribal, and non-governmental organizational levels, and to what extent PHAP has influenced alumni career paths (if at all). Information will be used to answer key program assessment questions, specifically: “Is PHAP a quality program?”, “Is PHAP an effective program?”, and “What is the impact of PHAP?” The results of the assessment will be shared in aggregate with PHAP program staff and published in peer reviewed journals and/or in non-scientific publications such as practice reports and/or fact sheets.

4. A statement detailing the impact the proposed collection will have on the respondent’s privacy

The proposed data collection will have little or no effect on the respondent’s privacy.

The PHAP Evaluation Team will follow procedures for assuring and maintaining security during all stages of data collection. Only the PHAP Evaluation team will collect/have access to data.

5. Whether individuals are informed that providing the information is voluntary or mandatory

As part of the introduction at the time of each data collection request, each participant will be provided with information describing the purpose of collecting the information and how the data will be used. Participants will be informed of the voluntary nature of their responses. There will be no effect on the participant should they not respond to the data collection request.

6. Opportunities to consent, if any, to sharing and submission of information

Prior to each information collection, the participants will be provided information on the intent of the data collection and will be given an opportunity to consent to the sharing and submission of information. All responses are voluntary.

7. How the information will be secured

All data will be stored in secured electronic files at CDC and will be accessible only to staff directly involved in the data collection. The spreadsheet that links unique identification numbers with personal identifiable information (i.e., name, email) will be maintained in a separate, password-protected record file on CDC file servers. This spreadsheet will be stored separately from all data collected from the PHAP Alumni Assessment to protect each individual’s privacy.

8. Whether a system of records is being created under the Privacy Act.

Parts of this data collection are subject to the Privacy Act. The existing applicable systems of Records Notice for this information collection is SORN 09-20-0161, “Records of Health Professionals in Disease Prevention and Control Training Programs.”

Estimates of Annualized Burden Hours and Costs

The estimate for burden hours is based on a pilot test of the PHAP Alumni Assessment by 5 public health professionals. In the pilot test, the average time to complete the assessment, including time for reviewing instructions and completing the assessment was approximately 7.5 minutes. Based on these results, the estimated time range for actual respondents to complete the assessment is 6-8 minutes. For the purposes of estimating burden hours, the upper limit of this range (i.e., 8 minutes) is used.

Estimates for the average hourly wage for respondents are based on the US Department of Labor (DOL) National Occupational and Wage Estimates for life, physical, and social science occupations.³ Based on DOL data, an average hourly wage of $33.37 is estimated for all 380 respondents. Table A-12 shows estimated burden and cost information. The total provided in this burden table accounts for one of group 1 Class 2012 being assessed twice over the course of the three year approval period in year 1 and again in year 3, totaling to 480 respondents.

Table A-12: Estimated Annualized Burden Hours and Costs to Respondents

Data Collection Instrument Form Name	Type of Respondent	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)	Total AnnualBurden Hours	Hourly Wage Rate	Total Respondent Costs
PHAP Alumni Assessment	PHAP Alumni	160	1	8/60	21	$33.37	$700.77

Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There will be no direct costs to the respondents other than their time to participate in each information collection.

Annualized Cost to the Government

The government costs include personnel costs for federal staff involved in project oversight, data collection, analysis, and reporting. These efforts involve a GS-14 Behavioral Scientist, a GS-12 Health Scientist, and a GS-9 Health Scientist. The total cost to the federal government is $11,378.40.

Table A-14: Estimated Annualized Cost to the Federal Government

Staff	Average Hours per Collection	Average Hourly Rate	Average Cost
GS-14 Behavioral Scientist	40	$48.90	$1,956
GS-12 Health Scientist	160	$34.90	$5,584
GS-9 Health Scientist	160	$23.99	$3,838.40
Total			$11,378.40

Explanation for Program Changes or Adjustments

This is a new data collection.

Plans for Tabulation and Publication and Project Time Schedule

Recruitment is scheduled for July for the purpose of aligning recruitment and data collection to the month associates graduate from PHAP (July). As the Alumni Assessment will be administered at 1 year, 3 years, and 5 years post-graduation, follow-up recruitment will start in July 1 year following each cohort’s graduation and at the 3 and 5 year post-graduation points.

Project Time Schedule

A.16 – 1 Project Time Schedule
Activity	Time Schedule
Email invitation sent to respondents	July 2015, July 2016, July 2017, October 2017
Data Collection	July-August 2015, July-August 2016, July-August 2017, October-November 2017
Analyses	September-October 2015, September-October 2016, September-October 2017, December-January 2018
Report Developed	November-December 2015, November-December 2016, November-December 2017, February-March 2018
Publication of Report	March 2016, March 2017, March 2018, June 2018

Reason(s) Display of OMB Expiration Date is Inappropriate

Display of the OMB Expiration Date is appropriate for this information collection.

Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

LIST OF REFERENCES – Section A

Centers for Disease Control and Prevention. About CDC: Mission, Role, and Pledge. Accessed 10.24.14; http://www.cdc.gov/about/organization/mission.htm

Centers for Disease Control and Prevention. Public Health Associate Program. Accessed 10.24.14; http://www.cdc.gov/phap/

Bureau of Labor Statistics. May 2013 National Occupational Employment and Wage Estimates United States for Life, Physical, and Social Science Occupations. Accessed 11.7.14; http://www.bls.gov/oes/current/oes_nat.htm

LIST OF ATTACHMENTS – Section A

Att. A Public Health Service Act

Att. B PHAP Alumni Assessment: Word Version

Att. C PHAP Alumni Assessment: Web Version

Page 12 of 12

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	CDC User
File Modified	0000-00-00
File Created	2021-01-25

PHAP ICR- Statement A_Revised 7 15