Justification for the Collection of Information for Planning Local Evaluations as part of the Personal Responsibility Education Program (PREP): Promising Youth Programs (PYP)
OMB Information Collection Request
0970 - 0356
Supporting Statement
Part A
August 2016
Submitted By:
Office of Planning, Research and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor Mary Switzer Building
330 C St, SW
Washington, D.C. 20201
Federal Project Officer:
Seth Chamberlain (COR)
Project Monitor:
Kathleen McCoy
A1. Necessity for the Data Collection
The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval to request information from grantees funded through the Personal Responsibility Education Program (PREP) to gather information about their local evaluations of teen pregnancy prevention programs. This information collection request (ICR) is submitted under the generic clearance for Formative Data Collections for Policy Research (0970-0356). Consistent with the goals of this generic clearance, the information collected through the templates for which we are seeking clearance seek to: (1) inform the development of OPRE research, (2) maintain a research agenda that is rigorous and relevant, and (3) inform the provision of technical assistance. Grantees are not required to complete these templates, but they will be strongly encouraged to complete them.
ACF and its contractor, Mathematica Policy Research, Inc., will collect information from grantees through two templates that ask grantees to summarize key information about their evaluation plans. This request will support initial reviews of evaluation design plans. This information is essentially for proper training and technical assistance to occur during the planning period. Related information collections for evaluation purposes will be submitted through a full information collection request. This current request is meant only for planning and development
To improve the life course of adolescents and reduce the risks related to sexual activity,
Congress authorized the Personal Responsibility Education Program (PREP) as part of the 2010 Affordable Care Act. It was reauthorized in 2015 for an additional two years of funding through the Medicare Access and CHIP Reauthorization ACT of 2015. PREP programs, which are sponsored by the Family and Youth Services Bureau (FYSB) within ACF, seek to reduce teen pregnancy and sexually transmitted infections (STIs) and prepare youth for adulthood. FYSB supports programs that exhibit evidence of effectiveness, innovative adaptations of evidence-based programs, and promising programs. They encourage, and in some cases require, grantees to conduct evaluations to inform the field’s current efforts and future programming for youth risk behavior prevention. FYSB awards PREP grants under four separate but related programs: (1) State PREP (SPREP), (2) Competitive PREP (CPREP), (3) Personal Responsibility Education Innovative Strategies Program (PREIS), (4) and Tribal PREP (TPREP).
The largest PREP program by funding amount is SPREP, which provides formula grants to states and U.S. territories. Beginning in 2010, states and territories were eligible to apply for formula grants, with funding calculated on the number of young people age 10 to 19. FYSB sponsors programs that target youth age 10 to 19 who are homeless, residing in foster care, adjudicated, living in rural areas or in geographic areas with high teen birth rates, or are members of racial or ethnic minority groups. The programs also support pregnant and parenting youth under age 21. States and territories are required to replicate evidence-based programs or substantially incorporate elements of evidence-based programs. Programs must educate youth about abstinence and contraceptive use and focus on at least three of the following adult preparation topics: healthy relationships, positive development, financial literacy, parent-child communication, and healthy life skills. Programs must ensure that content is culturally appropriate and medically accurate for participants. In 2016, 48 states and territories received a total of $43 million in State PREP allocations.
In states and territories that did not apply for State PREP, local service organizations may apply for grant funding through the Competitive PREP program. The affected states include Florida, Indiana, North Dakota, Texas, and Virginia as well as the territories of American Samoa, Guam, the Marshall Islands, the Northern Mariana Islands, and Palau.
PREIS supports the development of innovative strategies to prevent teen pregnancy. This program targets services to high-risk, vulnerable, and culturally underrepresented youth populations. This includes youth who are homeless, live in foster care, reside in areas with high teen birth rates, are members of racial or ethnic minority groups, or have HIV/AIDS. Programs may also target pregnant and parenting youth up to age 21. Projects must be based on some evidence of effectiveness, represent a significant adaptation of an effective program, or offer an innovative approach. Projects are also required to generate lessons learned by documenting the intervention, conducting process and outcome evaluations, and disseminating findings. The second cohort of PREIS grantees are expected to be funded in September 2016.
The overall goal of the Tribal PREP programs is to reduce teen pregnancy and birth rates and the spread of STIs for American Indian/Alaska Native (AI/AN) youth. Tribal PREP programs are expected to accomplish this goal by doing one of the following: replicating evidence-based programs (using culturally and linguistically appropriate adaptations as necessary); substantially incorporate elements of effective programs that have been proved on the basis of scientific research to change behaviors; or substantially incorporating elements or practices from programs that have demonstrated effectiveness within AI/AN tribal communities. All TPREP grantees are required to conduct a local evaluation, which can either be a descriptive or impact evaluation. The next cohort of Tribal PREP grantees are expected to be funded in September 2016.
In 2016, ACF contracted with Mathematica Policy Research to conduct the PREP: Promising Youth Programs (PYP) project, with the goal of supporting further development of the evidence base for teen pregnancy prevention programs. The project has three broad objectives:
1. Evaluate: The PYP project will provide local evaluation support (LES) to grantees funded in 2016 under the PREIS, TPREP, and SPREP programs. The LES tasks involve assisting grantees and their local evaluators with the development of evaluation plans, supporting grantees as they refine their evaluation plans, assisting with the implementation of plans by providing a community of learning, training, and other supports, and reviewing the final evaluation reports. The evaluation plans may involve randomized control trials, quasi-experimental designs, descriptive studies, and process evaluations. For the purposes of data collection, the evaluations will be grouped into two categories – 1) impact evaluations (which also are required to include a process study) and 2) descriptive evaluations. PREIS evaluations are required to do impact evaluations. TPREP grantees may conduct descriptive or impact evaluations. SPREP grantees are not required to conduct local evaluations, but may have proposed one; it is anticipated that local evaluation support would be provided to a subset of SPREP grantees with highly rigorous impact studies. Grantees who conduct a local impact evaluation will be required to collect a set of core measures (which we will seek OMB approval for in a separate ICR).
2. Innovate: The PYP project will also work to develop curricula for underserved youth (CUY) to address sexual health and other PREP-related priorities. The CUY tasks involve identifying target populations, developing theories of change, identifying organizations to implement curricula, providing support and monitoring to ensure fidelity of implementation, and collecting and analyzing information on experience to determine program successes and challenges.
3. Inform: The dissemination tasks under PYP involve ongoing dissemination throughout the project and dissemination of findings from the LES and CUY tasks. The dissemination approaches will address various target audiences, including federal staff, state officials and local program staff; program participants; parents; and local community members.
Under this generic information collection request (ICR), ACF seeks approval for two formative data collection instruments for the LES grantees used for the early planning activities. We will submit additional ICRs for the CUY data collection, data collection related to core measures for use by grantees conducting an impact evaluation, and any additional IC from the LES grantees used for later monitoring (for example,analysis plans and final reports).
Section 215 of the Medicare Access and CHIP Reauthorization Act of 2015 extends funding through FY2017 for PREP formula grants to states. The legislation mandates that the Secretary evaluate the programs and activities carried out with funds made available through PREP. To meet this requirement, FYSB and OPRE within ACF contracted with Mathematica Policy Research to conduct local evaluations of a subset of grantees in order to help meet this legislative requirement. This formative data collection is being done at the discretion of the agency, to gather prelimary formative information.
A2. Purpose of Survey and Data Collection Procedures
Overview of Purpose and Approach
One of the broad objectives of the PYP project is to provide evaluation support to grantees and their local evaluators. The purpose of the information collection instruments submitted through this request are to help the federal government and contractor staff provide evaluation technical assistance by providing the grantee feedback on their evaluation’s design and progress. This will help to ensure that local evaluations conducted are rigorous.
Research Questions
The information collected from the grantees will be used to ensure grantees design and implement evaluations that use methods that are of the highest quality and meet ACF’s standards for rigor related to the credibility, applicability, consistency, and neutrality of the evaluation. For example, we will examine the grantees’ proposed sample size and ensure they have adequate power to detect impacts. Additionally, we will ensure that the sample methodology ensures a strong contrafactual between the treatment and control group. In addition, impact evaluations must also be eligible to meet the HHS Teen Pregnancy Prevention Evidence Standards.1
Study Design
The information being collected using these instruments is for evaluation planning and technical assistance only. There is no independent study being conducted using the information collected. Therefore, there is no study design associated with this request.
Universe of Data Collection Efforts
The information collected by the contractor will help clarify the evaluation design and assist in the planning for the provision of relevant evaluation technical assistance. To achieve this, we propose collecting the following information from grantees, depending on the design they propose. Table A.1 shows which grantees will complete each template, based on the evaluation design proposed. We expect that 16 grantees will propose impact evaluations and 13 will propose descriptive studies.
Table A2.1 Use of Templates by Proposed Evaluation Design
Template |
Impact evaluations |
Descriptive studies |
Template 1: Impact evaluation design template |
X |
|
Template 2: Descriptive evaluation design template |
|
X |
Impact evaluation design template (Template 1). Grantees proposing an impact evaluation will complete the impact evaluation design template as part of the grant planning period (approximately the first nine months of the grant). This template will provide the grantee’s evaluation team with a structure to document the essential elements of the design to support clear documentation of their plans, and reflect any updates to the design since the application. The template will collect information on program services, target populations, study group formation, expectations for sample size, recruitment, and retention, outcomes, data collection methods, and plans for an implementation/process study. The use of a structured template will allow the contractor to provide effective evaluation technical assistance prior to the start of the evaluation to adjust the design as necessary to ensure the level of rigor required by ACF is met.
Descriptive evaluation design template (Template 2). Grantees that propose a descriptive study will complete the descriptive evaluation design template. This template is similar to the impact evaluation design template, but focuses on the types and quality of data collected in a descriptive study, which could address process or outcome questions. The use of a structured template will allow the contractor to provide effective evaluation technical assistance prior to the start of the evaluation to adjust the design as necessary to ensure the level of rigor required by ACF is met.
A3. Improved Information Technology to Reduce Burden
ACF and its contractors will employ information technology as appropriate to reduce the burden of respondents who agree to participate. This data collection effort will make use of electronic templates for completion and electronic submission.
A4. Efforts to Identify Duplication
There are no other sources of information that would allow us to gather the discussed formative information from the ACF-funded PREP programs for the upcoming round of grant funding. We propose to use templates that have successfully been used in prior studies involving similar populations and programs. No unnecessary information is being requested of program staff or participants from the upcoming round of PREP grants. We will ensure that we do not collect information that is available elsewhere. None of the instruments will ask for information that can be reliably obtained through other sources.
A5. Involvement of Small Organizations
The potential exists for data collection activities to affect small entities associated with the grantee. PREP grantees may conduct evaluations led by local evaluators affiliated with small organizations. Grantees may task the local evaluator with the collection of some or all of the data requested. Proposed data collection efforts are designed to minimize the burden on all organizations involved, including small businesses and entities, by collecting only critical information through the use of standardized templates.
A6. Consequences of Less Frequent Data Collection
The purpose of each information collection instrument included in this submission is described in Item A2, above. Not collecting information using these instruments would limit the government’s ability to document the planning processes of its grantees and to provide the Technical Assistance needed.
Specifically, without the information collected through both evaluation design templates at the start of the grant, the government and contractor would not be able to determine whether the proposed design will meet the grantee’s research objectives as well as the standards of rigor set by ACF. We expect grantees will need to update their designs three times during their planning period. It is common during the planning and early stages of evaluation that details of the design and implementation change due to unexpected challenges.
A7. Special Circumstances
There are no special circumstances for the proposed data collection efforts.
A8. Federal Register Notice and Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on September 15, 2014, Volume 79, Number 178, page 54985, and provided a sixty-day period for public comment. During the notice and comment period, no comments were received, which is attached.
ACF consulted staff from Mathematica Policy Research, and their subcontractor the Center for American Indian and Native American Health at the University of Colorado, Denver when preparing the templates. See Attachment A for a list of the persons consulted.
A9. Incentives for Respondents
No incentives for respondents are proposed for this information collection.
A10. Privacy of Respondents
Information collected from both templates will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law.
As specified in the contract, Mathematica (the Contractor) shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements.
Information will not be maintained in a paper or electronic system from which they are actually or directly retrieved by an individuals’ personal identifier.
A11. Sensitive Questions
There are no sensitive questions in this data collection.
A12. Estimation of Information Collection Burden
Newly Requested Information Collections
The burden table provides the estimated reporting burden calculations for the two instruments included in this request.
The number of responses represents the total number of times a respondent will complete each instrument over a one year period and the burden hours are averaged across all respondents. Burden hours are rounded to the nearest hour. Assumptions by instrument are as follows:
Impact evaluation design template: Across the planning year, a maximum of 16 grantees will complete the impact evaluation design template. Respondents will complete/revise the template up to three times. Respondents first will complete the full template, which we estimate will take eight hours. Respondents then will be asked to provide up to two revisions to elaborate or clarify responses. We anticipate that the first revision may take respondents up to four hours and the second revision may take up to two hours. Averaged over the three responses, the average burden per response is five hours.
Descriptive evaluation design template: Across the planning year, a maximum of 13 grantees will complete the descriptive evaluation design template. Respondents will complete/revise the template up to three times. Respondents first will complete the full template, which we estimate will take eight hours to complete. Respondents then will be asked to provide up to two revisions to elaborate on or clarify responses. We anticipate that the first revision may take respondents up to four hours and the second revision may take up to two hours. Averaged over the three responses, the average burden per response is five hours.
Table A12.1 Total Burden Requested Under this Information Collection Request
Instrument |
Total Number of Respondents |
Number of Responses Per Respondent |
Average Burden Hours Per Response |
Total Burden Hours |
Average Hourly Wage |
Total Cost |
||
Template 1: Impact evaluation design template |
16
|
3 |
5 |
240 |
$33.38 |
$8,011.20 |
||
Template 2: Descriptive evaluation design template |
13 |
3 |
5 |
195 |
$33.38 |
$6,509.10 |
||
Estimated burden |
435 |
|
$14,520.30 |
|||||
Total Annual Cost
The total burden cost is estimated to be $14,520.30. For all cost calculations, we estimate the average hourly wage for program directors and managers to be the average hourly wage for “Social and Community Services Manager” ($33.38), taken from the U.S. Bureau of Labor Statistics, Occupational Employment Statistics, 2015.
A13. Cost Burden to Respondents or Record Keepers
These information collection activities do not place any additional costs on respondents or record keepers other than those described above.
A14. Estimate of Cost to the Federal Government
The total cost for the data collection activities under this current request, including the development of the templates and reviewing the design plans will be $261,794. Annual costs to the Federal government will be $87,265 for the proposed data collection.
A15. Change in Burden
This is a new data collection. There are no changes or adjustments.
A16. Plan and Time Schedule for Information Collection, Tabulation and Publication
The schedule for data collection is shown below in Table A16.1. All dates are dependent on OMB approval of this ICR.
Table A16.1 Schedule for PYP Evaluation Monitoring Data Collection
Activity |
Date |
Grantee awards |
September 30, 2016 |
Impact evaluation design template |
October 2016-June 2017 |
Descriptive evaluation design template |
October 2016-June 2017 |
.
No publications are expected from this data collection.
A17. Reasons Not to Display OMB Expiration Date
All instruments will display the expiration date for OMB approval.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are necessary for this information collection.
| File Type | application/msword |
| File Title | OPRE OMB Clearance Manual |
| Author | DHHS |
| Last Modified By | Windows User |
| File Modified | 2016-08-25 |
| File Created | 2016-08-25 |