Supporting Statement B - SARHM Youth Telephone Interviews

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Self-Regulation Training Approaches and Resources to Improve Staff Capacity for Implementing Healthy Marriage Services for Youth (SARHM)

Formative Data Collections for Program Support

0970 - 0531

Supporting Statement

Part B

September 2020

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Aleta Meyer and Caryn Blitz

Part B

B1. Objectives

Study Objectives

The objectives of the focus groups are to answer two research questions.

1. How do adolescents and youth participating in ACF-funded HMRE programs describe self-regulation in their own words?

2. How salient do adolescents and youth participating in ACF-funded HMRE programs see self-regulation skills to the events and activities in their lives?

Generalizability of Results

This study is intended to present an internally-valid description of the service population in chosen sites, not to promote statistical generalization to other sites or service populations.

Appropriateness of Study Design and Methods for Planned Uses

For this nonsubstantive change request, SARHM is requesting to shift the mode of data collection from focus groups to individual telephone interviews. Individual interviews are a standard qualitative data collection practice. They will enable the study team to be able to speak to as many youth as possible within a short timeframe, because they can be flexibly scheduled.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.  

B2. Methods and Design

Target Population

This will be a multisite study. In September 2015, ACF awarded five-year grants to 46 HMRE grantees; 31 of these grantees serve youth. The target population are youth that participate in HMRE programs. Youth in these programs are between the ages of 14-24. HMRE programs disproportionately serve underrepresented, high-risk youth including racial and ethnic minorities, youth living in high-poverty neighborhoods, and teenage parents (Scott et al. 2017). These disadvantaged youth may be enrolled in school or they may be served in a community setting. We plan to involve up to three sites in the study, with a maximum of 25 youth between the sites.

Sampling and Site Selection

The SARHM project will conduct telephone interviews with youth who participated in an HMRE program or services with a HMRE program partner organization at one of the three following sites:

• Youth and Family Services, Inc. (Rapid City, South Dakota)

• Public Health Institute (Central Valley, California)

• Bethany Christian Services (Grand Rapids, Michigan)

These three sites were chosen from among the 31 youth-serving grantees referenced above.

To select sites from this pool, we first asked federal program specialists to provide input on programs’ grant performance and their perceptions of programs’ capacity to recruit a focus group of 8-10 youth. The project team, including the contractors and federal Contracting Officer’s Representatives, then reviewed the list of recommended programs and purposively chose five to contact to learn more about their programs and interest and willingness in hosting a focus group. Out of the five, we selected three programs to engage in further discussions.

We selected these three to reflect a range of program types and service populations, using data available in grant applications and other program documents to identify a purposive sample of programs. In particular, we sought to include programs serving high-risk and disadvantaged youth in school-based and community locations; programs that served younger youth (14-18 years old) and older youth (18-24 years old); and diversity in the demographic characteristics of the service population. Our list included two school-based programs, one community-based program, and one that operates in both school- and community-based settings. Together, the programs serve youth from ages 14-24. The programs operate in a mix of urban, suburban, and rural settings and serve varied populations of vulnerable and at-risk youth, including underrepresented racial and ethnic minorities, disconnected youth, and youth living in high-poverty areas.

We reached out to these three programs to ascertain their interest and capacity to participate in the study. Based on these calls, we chose the three aforementioned sites, which had the greatest interest in participating and ability to recruit youth to participate in a focus group. After the COVID-19 pandemic forced the cancellation of in-person focus groups, we held discussions with the sites to confirm that they had access to program participants to assist with recruitment for telephone interviews.

B3. Design of Data Collection Instruments

Development of Data Collection Instrument(s)

The contract teams (Mathematica and Public Strategies) and ACF worked together to design the telephone interview protocol (Instrument 1). The instruments were reviewed internally by the project team to streamline them and ask only questions necessary to achieve the objectives of data collection. All of the questions were created specifically for the study; there are no scales or items that required psychometric testing. The instrument was adapted for a telephone interview after the project team determined the need to conduct data collection virtually. It uses the same questions and asks about the same constructs as Instrument 1, with minor, nonsubstantive adaptations made to fit the context of an individual telephone interview instead of a focus group (for example, changes to the introductory language and to formatting). The instrument was shortened, since virtual focus groups and interviews will be shorter than the originally planned in-person data collection. Additionally, a few edits were made to the interview protocol to more clearly reflect the full scope of self-regulation (for example, feeling pressure and resisting temptation when wanting to spend time with good friends [added ‘when wanting to spend time with good friends’]; sticking to a long-term plan for reaching goals that are connected to each other [changed ‘long-term goal’ to ‘long-term plan for reaching goals that are connected to each other’]).

B4. Collection of Data and Quality Control

Two Mathematica SARHM project team members will collect the data. Mathematica will work with selected sites to recruit youth to participate in the telephone interviews.

The data collection mode is telephone interviews. The Mathematica researcher who will lead the interviews has extensive experience interviewing youth and has undergone interviewer training. The interviews will be audio-recorded, with participants’ permission.

No data evaluation activities are currently planned. The same Mathematica project team members will conduct all of the interviews.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

The telephone interviews are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. Response rates will not be calculated or reported.

NonResponse

As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated.

B6. Production of Estimates and Projections

We will not be producing estimates or weights for this project.

B7. Data Handling and Analysis

Data Handling

The interviews will be audio recorded with respondents’ permission; the audio recordings will be used as needed to supplement, correct, or clarify information in notes taken by a notetaker.

Data Analysis

The project team will use standard qualitative procedures to analyze and summarize information from the telephone interviews. Analysis will involve coding and theme identification. We will begin analysis with an initial set of codes based on the instrument and allow for the creation of additional codes based on emerging themes. No other data sources will be used in this study. The results will be used to describe themes, messages, key words, and phrases that focus group respondents use to describe self-regulation processes. They will be summarized in a memo to ACF and used to inform messages and training materials designed to enhance ACF-funded HMRE programs’ capacity to support youth self-regulation skill development by providing examples of language programs can use in working with youth.

Data Use

The information collected under this clearance will not be the primary subject of any published ACF reports; however information may be made public through future training materials. This study is intended to present an internally-valid description of the service population in chosen sites, not to promote statistical generalization to other sites or service populations. When necessary, results will be labeled as exploratory in nature.

B8. Contact Person(s)

For questions about these materials, please contact:

Scott Baumgartner

Task Leader, Mathematica

SBaumgartner@mathematica-mpr.com

Attachments

SARHM Instrument 1

Appendix A: Legislative Authority

Sources

Scott, Mindy, Elizabeth Karberg, Ilana Huz, and Mary Jo Oster. “Healthy Marriage and Relationship Education Programs for Youth: An In-Depth Study of Federally Funded Programs.” OPRE Report #2017-74: Washington, DC. Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2017.

7