Ss_a

TPP Tier 1B Design and Implementation Study Supporting Statement A

Supporting Justification for OMB Clearance of Teen Pregnancy Prevention Tier 1B

Design and Implementation Study

Part A: Justification for the Collection of Implementation Data

May 2016

Part A Introduction

Under the Office of Adolescent Health’s (OAH) Tier 1B grant program, 50 grantees received five-year cooperative agreements in July 2015 to implement evidence-based Teen Pregnancy Prevention (TPP) programs to scale in communities with the greatest need. The grant strategy focuses on providing evidence-based programs (EBPs) in multiple settings, mobilizing communities, and establishing and maintaining linkages to youth-friendly health care services to reduce teen pregnancy and existing disparities on a community-wide scale. These community-wide efforts build on earlier funding efforts in which single evidence-based programs were implemented in single settings. OAH is interested in learning how grantees funded under this grant program are scaling-up these efforts to strengthen and expand the reach of evidence-based TPP programs. The TPP Tier 1B Design and Implementation Study will add valuable information to what we are learning about the implementation of TPP programs. OAH is explicitly interested in understanding and documenting how the 50 Tier 1B grantees are implementing TPP projects to scale in their communities and the challenges they have faced in this process. This package covers the review and documentation of early grantee approaches and experiences.

1. Circumstances Making the Collection of Information Necessary

Despite great progress in reducing rates of teen pregnancy and births, large disparities in teen births exist among racial and ethnic groups, socio-economic status, and across states and within communities (Cox et al., 2014). There is some recent evidence to suggest that more than educational programming is needed to achieve significant improvements in the lives of youth (CDC, 2015). The TPP Tier 1B grant program acknowledges the successes and limitations of implementing single evidence-based programs in single settings, and expands the strategy to one that reaches youth in multiple settings and at multiple points during their adolescence. By focusing on mobilizing whole communities, it is more consistent with the recommendations of the CDC and more generally with a public health model. To understand and document the experiences of grantees replicating TPP models to scale in high-need communities, we propose to collect implementation information on each grantee project.

The work of this study includes a descriptive assessment and documentation of how the 50 projects awarded funding under the 2015 Tier 1B grant program are scaling their programs (i.e., the strategies that were planned and implemented to expand the reach of evidence-based programs, mobilize communities, and establish and maintain linkages to youth-friendly health care). More intensive discussion with a subset of grantees, their partners, and constituents will provide detailed information about the implementation and scaling-up experiences of projects that represent the diversity of the grantees in scale-up strategies and settings.

Legal or Administrative Requirements that Necessitate the Collection

Under the authority of Division H, Title II of the Consolidated Appropriations Act, 2014 (Public Law No. 113-76), and the Continuing Resolution thus far for 2015 (Public Law No. 113-164), the Office of Adolescent Health made available $60 million in funds to support the OAH Teen Pregnancy Prevention Program. The goal of the funds is to have a significant impact on reducing rates of teen pregnancy and existing disparities by replicating evidence-based programs to scale in at least 3 settings in communities and with populations at greatest need. The funds support the HHS Strategic Goal to “Put Children and Youth on the Path for Successful Futures.” Under section 241 of the Public Health Service Act, $6,800,000 was made available to carry out evaluations of teen pregnancy prevention approaches. The TPP Tier 1B Design and Implementation Study is a key piece of OAH’s broad and ongoing effort to comprehensively evaluate all of its teen pregnancy prevention funding efforts as required by the legislation.

Objectives of the TPP Tier 1B Design and Implementation Study

The study as a whole is intended to: document and describe the implementation of evidence-based teen pregnancy prevention programs to scale in communities with the greatest need; strengthen grantee project-level evaluation designs that will address questions about effects on a set of community-level outcomes related to youth well-being; and design a rigorous quantitative study that will examine the overall effectiveness of the grant program.

The implementation data collection requested for clearance in this package will address the following primary research questions:

• How are grantees and their partners using the core elements of the Tier 1B grant strategy to take programs to scale?

• How did grantees and their partners plan to and initially implement EBPs in multiple settings to reduce teen pregnancy and address disparities at the community level?

• How did grantees and their partners plan and begin to mobilize their communities to prevent teen pregnancy and promote positive youth development?

• How did grantees plan and begin to establish and maintain community-wide linkages and referrals to youth-friendly health care services?

Reviewing and reporting on TPP Tier 1B program planning and implementation will include the following activities:

• A review of the 50 grantees’ original applications for Tier 1B grants.

• Selection of up to 15 grantees with a representative set of implementation strategies for participation in site visits.

• Telephone interviews with the population of 50 grantee organizations and up to two implementation partner organizations per grantee.

• Site visit interviews and focus groups with grantees and other stakeholders.

• Analysis and reporting of qualitative data.

Implementation Data Collection Instruments

We propose two data collection instruments for the phone interviews (one for grantee staff leading the project, and one for partner organization staff) and a general topic guide for the site visit interviews and focus groups. We anticipate that the unique features of individual grantee communities selected for the site visits and the roles of grantee staff, partners, youth advisors, and other stakeholders will vary between and within grantee communities. To collect the most appropriate information on each project’s implementation, site visit team leaders will customize the topic guide for each grantee community and the associated interviews and focus groups. Attachment A provides an overall guide to the data collection topics and associated data sources. Attachment B contains the phone interview guides for grantee and partner staff.

Through the work of this study, OAH will ultimately have a set of comprehensive documents that provide important implementation and evaluative information critical to understanding efforts to take programs to scale and ultimately address the needs of youth in high-need communities and reduce disparities in rates of teen pregnancies and births. These documents will include:

• A report synthesizing findings from the analysis of application review and telephone interview data: the report will include both a cross-grantee summary and individual grantee profiles. This report will clearly document how Tier 1B projects were designed, and how key scale-up mechanisms were used to expand impact on teen pregnancy and disparities.

• A set of project summaries detailing the findings of each site visit (one for each of the up to 15 grantee projects). These will be accessible to groups such as Tier 1B grantees, others providing youth sexual health services and other prevention services for youth, other public health professionals, researchers, Federal policymakers, and the general public.

2. Purpose and Use of Information Collection

The TPP Tier 1B design and implementation study will collect data on program planning and implementation. This will include information about: processes for mobilizing the community; establishing partnerships; determining program models and implementation settings; activities during the planning and piloting year; description of programs being implemented, by whom, where and to what populations; how fidelity and quality monitoring are used; strategies for recruitment and retention of youth participants; how programs are ensured to be medically accurate, age appropriate, culturally and linguistically appropriate, and inclusive of LGBTQ youth; ensuring safe and supportive environments for youth; how linkages and referrals to youth-friendly health care services are established and maintained; strategic dissemination activities; sustainability planning; training and public education activities undertaken by grantees; and other successes, best practices, lessons learned and barriers relevant to program implementation.

Information on these topics will be collected from existing program documents (including the grant applications submitted by each of the 50 TPP Tier 1B grantees), as well as individual telephone interviews with grantees and their implementation partners (all 50 projects), and data collection during site visits (up to 15 projects). The on-site data collection will include: semi-structured key informant interviews with grantee staff and staff from partner organizations; collection of any additional documents provided by grantees or their partners for later review; semi-structured observations of activities, classes and other services; and focus groups with youth and community stakeholders. Attachment A includes the topic guide and corresponding data sources.

The data will be used for two purposes. First, the information will enable the study team to produce clear, concise profile descriptions of each implementation. The report will also provide a cross-grantee summary of grantee strategies for replicating evidence-based programs to scale and reaching community-wide saturation. This report will be critical in understanding the universe of strategies employed to reach the goal of serving and mobilizing an entire community to reduce teen pregnancy and disparities. Second, the data will address the challenges and opportunities encountered by grantees and how grantees addressed or leveraged these. The project summaries of the site visits will illustrate in detail a variety of approaches and strategies for scaling up evidence-based approaches to teen pregnancy prevention, describe the drivers of implementation, and highlight topics of particular interest to OAH. The data collection and analysis will also provide information critical to implementing subsequent similar community-wide initiatives.

3. Use of Improved Information Technology and Burden Reduction

The data collection plan reflects sensitivity to issues of efficiency, accuracy, and respondent burden. Where feasible, information will be gathered by extracting it from existing documents. Protocols for telephone interviews and in-person interviews and group discussions during site visits will be customized for each site to focus on information that is relevant for that site and that could not be obtained from documents. Conducting telephone interviews in advance of the site visits will allow not only the collection of consistent cross-grantee information, but also a reduction in questions necessary to discuss with grantees and partners during site visits.

Zero percent of responses will be collected electronically, as all interviews and focus groups will be conducted by phone or in person.

Improved information technology will be used when appropriate. For example, when program information or documents can be sent electronically, we will not request a hard copy of the documents.

4. Efforts to Identify Duplication and Use of Similar Information

The information collection requirements for the TPP Tier 1B Design and Implementation Study have been carefully reviewed to determine what information is already available from existing studies and what will need to be collected for the first time. Although prior studies contribute to our understanding of teenage sexual risk behavior and past efforts to reduce it, OAH does not believe they provide sufficient information on community-wide efforts to policymakers and stakeholders. In addition, Congress is supporting evaluations, including longitudinal evaluations, of adolescent pregnancy prevention approaches. The data collection for the TPP Tier 1B Design and Implementation Study is an essential step in providing this information.

5. Impact on Small Businesses or Other Small Entities

Initiatives and programs in some sites may be operated by or in collaboration with small community-based organizations. The data collection plan is designed to minimize burden on such organizations by focusing interviews with their staff on their direct role in the intervention and its development or planning.

6. Consequences of Collecting the Information Less Frequent Collection

It is essential to collect data directly from those responsible for implementing the program to understand and assess the scale-up efforts. We propose collecting these data once to capture implementation experiences during the planning and early implementation period. There are no legal obstacles to reduce the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for the proposed data collection.

8. Comments in Response to the Federal Register Notice/Outside Consultation

The 60-day notice was published in the Federal Register on April 19, 2016. A copy of that notice is found in Attachment F. At this time there are no comments or responses to questions. The text of the 30-day notice published in the Federal Register can be found in Appendix G.

Below, we provide the names and contact information of the persons consulted in the drafting and refinement of the implementation study protocols (2016):

Meredith Kelsey, Ph.D

Abt Associates

55 Wheeler Street

Cambridge, MA 02138

(617) 520-2422

Kimberly Francis, Ph.D

Abt Associates

55 Wheeler Street

Cambridge, MA 02138

(617) 520-2502

Jean Layzer, M.A.

Belmont Research Associates

42 Fairmont Street

Belmont, MA 02478

(617) 484-8189

Lesley Freiman, MPP

Abt Associates

55 Wheeler Street

Cambridge, MA 02138

(617) 520-2334

Carolyn Layzer, Ph.D

Abt Associates

55 Wheeler Street

Cambridge, MA 02138

(617) 520-3597

Amy Farb, Ph.D

Office of Adolescent Health

Office of the Assistant Secretary for Health

U.S. Department of Health and Human Services

1101 Wootton Parkway, Suite 700

Rockville, MD 20852

(240) 453-2836

9. Explanation of any Payment/Gift to Respondents

No payment or gift will be made to organization staff and adult community members for being interviewed by phone or during site visits. Gift cards of $15 will be offered to Youth Leadership Council (YLC) members participating in the focus groups to compensate them for their time when they could otherwise be engaged in activities such as care of siblings, working, homework, etc.

10. Assurance of Confidentiality Provided to Respondents

Abt Associates will secure IRB approval for the collection of implementation data. Privacy protections are embedded in the study design. Implementation study respondents will receive information about how their information will be kept private when arrangements are made for meeting with them, and this information will be repeated as part of the site visit field staff’s introductory comments during site visits (see Attachment C for an example of these introductory comments). This information will also be included in initial introductory e-mails prior to telephone interviews (see Attachment D for introductory e-mail template). The study will not be asking any sensitive questions. Site visit field staff will be informed about privacy and confidentiality procedures during training and will be prepared to describe them and to answer questions raised by local program staff.

Youth under age 18 who participate in the focus groups must have parental permission to participate, in addition to providing their own assent. Youth invited to a focus group as part of the implementation study will be asked to complete an assent form before the focus group is conducted and parents of youth under 18 will be asked to sign a permission form. This form will state that answers will be kept private, that youths’ participation is voluntary, that they may refuse to participate, and that identifying information about them will not be released or published. The youth assent form and parental permission form are provided in Attachment E.

11. Justification for Sensitive Questions

The phone interview and site visit protocols do not contain sensitive questions. Interviews and focus groups with grantee and partner organization staff and CAG members will focus on the components of the community-wide teen pregnancy prevention initiative and the experiences of staff and their organizations in implementing or supporting them. Focus groups with youth participating in the YLCs will focus on the role of the YLC in the initiative, YLC members’ perspectives on youth-adult partnerships and community mobilization, challenges encountered in their YLC role, and how to address them.

12. Estimates of Annualized Hour and Cost Burden

A single data collection per individual is anticipated. Based on experience with similar protocols and respondents, ninety minutes is allotted for each telephone conversation (one grantee leader per project and two grantee partners per project). One hour is allotted for each site visit interview (8 interviews per project) and for each focus group (3 focus groups per project, assuming 8 participants each). The total burden for this data collection is estimated to be 465 hours, with a total cost of $11, 312.50 (please see Tables 12.1 and 12.2).

Table A12.1. Calculations of Burden Hours for Implementation Study Participants

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Grantee director (telephone)	Attachment B	50	1	90/60	75
Other grantee staff	Attachment A	60	1	1	60
Partner director (telephone)	Attachment B	100	1	90/60	150
Partner staff	Attachment A	60	1	1	60
Youth Leadership Council members	Attachment A	80	1	1	80
Community Advisory Group Members	Attachment A	40	1	1	40
Total		390			465

Table A12.2 Estimated Annualized Burden Costs for Implementation Study Participants

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Grantee director (telephone)	75	$37.50	$2,812.50
Other grantee staff	60	$20	$1,200
Partner director (telephone)	150	$30	$4,500
Partner staff	60	$20	$1,200
Youth Leadership Council members	80	$7.50	$600
Community Advisory Group Members	40	$25	$1,000
Total	465		$11,312.50

13. Estimates of other Total Annual Cost Burden to Respondents or Recordkeepers/ Capital Costs

These information collection activities do not place any additional cost on respondents.

14. Annualized Cost to Federal Government

Data collection will be carried out by Abt Associates, under contract with OAH to conduct the TPP Tier 1B Design and Implementation Study. The total cost for collecting the implementation data is $960,956.24, and the annual cost is 480,478.12.

15. Explanation for Program Changes or Adjustments

No program adjustments are anticipated based on this data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

Phone Interviews: Cross-grantee report and grantee profiles

Phone interview data for each grant project for the individual grantee profiles as well as across grantees will be summarized to highlight the range of implementation characteristics and strategies and common themes.

The interview team will code certain basic information about the grants into binary, continuous, or categorical variables as appropriate. The coding will be done by the note-taker, and a system of quality control checks will be implemented to ensure accuracy and consistency. The grantee profiles will largely consist of these data; for the cross-grantee assessment, these data will be summarized using descriptive statistics (e.g., means, frequencies, or distributions) to address relevant data elements.

A qualitative analytic approach will be used to identify themes emerging from open-ended responses and thoroughly describe the key data elements of interest. The study team will use NVivo qualitative software to systematically code and summarize all qualitative data. The study team will determine most topics and coding categories prior to coding, but we expect codes will be revised and expanded after initial review of the data. To ensure accuracy and consistency, senior qualitative researchers will review the coded interview notes, reconcile any differences in coding, and confirm the thematic findings.

Findings from the analysis of telephone interview data will be synthesized into a report. The report will include both a cross-grantee summary and individual grantee profiles to provide OAH and stakeholders with a clear understanding of how Tier 1B projects were designed, and how key scale-up mechanisms were used to expand the impact on teen pregnancy and reduce disparities at the community level. The cross-grantee summary portion of the report will include quantitative information such as the total number of implementation sites, providers, types of settings, programs used, and youth served as well as qualitative information such as summaries of the community mobilization and program selection processes, target populations, how programs are ensured to be medically accurate and culturally and linguistically appropriate, and plans for sustainability. Individual grantee profiles will include general implementation features including, but not limited to, the name of the grantee, any sub-awardees or partners and their roles, program models used, target populations, implementation settings, number of implementation sites, number of providers, estimated number of youth to be served across the grant period, and other characteristics that will round out a succinct but informative picture of each project.

Project Summaries

For the site visit data, a Multiple Case Study Analysis (Stake, 2006) will be conducted using OAH priority topics as sensitizing concepts. Each case will illustrate one community-wide example of how the Tier 1B program is working by providing extensive detail about topics across the selected projects and highlighting topics of particular strategic interest.

To efficiently analyze data collected, the study team will use NVivo software to organize and code qualitative data, artifacts, and documents. As in traditional qualitative analysis, coding of pre-selected categories and development of themes that emerge from the data can be done with NVivo, but two advantages of using this software are that it facilitates inter-rater agreement checks and ad hoc data queries.

Publishing and Dissemination

Each of these reports, as well as derivative products or excerpts (e.g., special topics briefs, articles, presentations), will be provided publically and targeted to specific audiences, such as Federal and local grantee staff, nonprofit service providers, and researchers. More traditional approaches (e.g., journal publications, conference presentations, and policy briefs) are frequently used for research/policy-aware audiences, while newer approaches (e.g., webinars and the use of infographics, data visualizations, social media, whiteboarding, digital scrapbooks, and explanatory videos) often appeal to a broader audience. The project will most likely leverage multiple strategies, given audiences that respond differently to written text versus visual images.

Project Time Schedule

OAH is requesting a three-year clearance. Data collection (phone interviews and site visits) is expected to begin in November 2016 and continue through July 2017. Reports from the telephone interviews will be completed in June 2017, and project summary reports will be completed in August 2017. Below is a schedule of the data collection efforts for the implementation study:

Table A16.1. Timeline for Use of Implementation Study Protocols

Instrument	Date of 60-Day Submission	Date of 30-Day Submission	Date Clearance Needed	Date for Use in Field
Instrument 1: Site visit topic guide	April 2016	May 2016	September 2016	November 2016
Instrument 2: Telephone interview guide	April 2016	May 2016	September 2016	November 2016

17. Reason(s) Display of OMB Expiration Date is Inappropriate

All protocols, parent permission and assent forms will display the OMB number and the expiration date.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

12