Non Sub Change Supporting Memo

To: Elaine Soohoo and Jamie Wilson

Office of Information and Regulatory Affairs (OIRA)

Office of Management and Budget (OMB)

From: Mary Mueggenborg

Office of Planning, Research and Evaluation (OPRE)

Administration for Children and Families (ACF)

Date: December 16, 2022

Subject: Nonsubstantive Change Request – Supporting Youth to be Successful in Life (SYSIL) Study (OMB #0970-0574)

This memo requests approval of nonsubstantive changes to the approved information collection, Supporting Youth to be Successful in Life (SYSIL) Study (OMB #0970-0574).

Background

OMB approved the impact and implementation studies and instruments for the OPRE Study, Supporting Youth to be Successful in Life (SYSIL) on July 27, 2021. Baseline data collection with youth for the impact study began in September 2021. Data collection for the 6-month and 12-month follow-up surveys began in April 2022 and October 2022, respectively. Two non-substantive changes were requested and approved in December 2021 (expansion of sample) and June 2022 (additional reminders to participants and informational documents). The approved collection includes contacting youth periodically to remind them to complete a survey and to notify us of any contact information updates.

After beginning outreach for the 6-month and 12-month surveys, we received feedback from youth that some questions on the survey were emotionally draining and as a result, some youth do not complete survey. To ensure that the research process is trauma-informed for youth, we want to be mindful of these triggers and honor this request. Youth are informed that they can skip any questions that they do not want to answer and many questions offer a response option of “choose not to answer.” To further minimize discomfort and, hopefully, increase a youth’s ability to complete the survey, we propose removing the survey items reported as particularly sensitive.

The approved collection includes contacting youth periodically via text and email to notify Mathematica of any contact information updates. We would now like to revise this outreach to incorporate four short contact information requests to provide an opportunity for youth to confirm their current contact information and continue to engage and familiarize youth with the SYSIL study.

Overview of Requested Changes

This memo requests the following nonsubstantive changes:

• Remove and reorder select survey items in Sections D (“Relationships and communication”) and E (“Social and emotional wellbeing”) to reduce youth-reported emotional burden to respondents. Section K (“Empowerment”) will be moved between Sections D and E (Instrument 1).

• Revise outreach messages to youth requesting updated contact information (Appendix C).

• Implement four short contact information requests to be sent to youth to ensure their contact information remains accurate and to continue to familiarize respondents with SYSIL (Instrument 5).

Removal and reordering of emotionally burdensome survey items in follow-up surveys

We propose adjusting the questions on the 6-month, 12-month, and 24-month follow-up surveys to reduce the potential for emotional discomfort for respondents. Caseworkers, phone interviewers, and youth reported that questions on the survey have been upsetting for some respondents, in some instances causing them to not complete the survey. Specifically, these individuals identified Sections D (“Relationships and Communication”) and E (“Social and Emotional Wellbeing”) as warranting necessary changes due to their content. Table 1 identifies the specific items we propose deleting, as well as the rationale for deleting the question. The proposed revised survey is found in the attached survey instrument (Instrument 1).

We anticipate removing the proposed items will decrease the burden to complete each survey by 5 minutes (0.08 hours).

Table 1. Proposed survey items to drop

Survey item1	Reason for dropping
D6 Indicate how much or how little each statement feels like you. “There are people in my life who…” “I have someone in my life who…” “I feel lonely…” “I feel no one loves me…”	Length— this item asks youth to respond to 17 statements about support and relationships with others, which may be burdensome. Similar item exists—We believe items D1-D5 can sufficiently capture whether youth have supportive connections and the type of connections. D6 also includes statements about feeling lonely and unloved, which we believe item E4 sufficiently captures. Concern—For youth who may have limited support or are feeling lonely, it may be emotionally draining to report on that repeatedly across 17 statements.
D8 Are the following statements like me? “My relationships are free from hitting, slapping, shoving, being made fun of, or name calling…” “I can deal with anger without hurting others or damaging things…” “I think about how my choices impact others…”	Concern—this item asks about physical violence, a potentially triggering topic. Distal outcome—Pathways does not specifically target physical violence. We do not believe this item will be sensitive to the intervention. If Pathways places youth in safer environments where they are less likely to experience violence, then we will capture the favorable effect on safer environments in Section C of the survey.
E2 Sometimes problems can get in the way of doing everyday activities. How hard is it for you to do each of the following? “Get along with adults outside the family…” “Control your emotions and stay out of trouble…” “Express your feelings…”	Similar item exists—this item is intended to capture communication skills, self-control, and interpersonal relationships with adults. We believe E3 can sufficiently capture this construct.
E5 Mark whether, during the past month, you felt this way never, once or twice, about once a week, 2 to 3 times a week, almost every day, or every day. “How often did you feel… Happy? Satisfied with life? That you had warm and trusting relationships with others?”	Length—this item asks youth to respond to 14 statements that capture aspects of mental health. Similar item exists—E4 and E5 are both intended to capture mental health. We believe E4 can sufficiently capture this construct. Concern—This item may be particularly burdensome if youth have to repeatedly indicate a lack of happiness, satisfaction, trusting relationships, a sense of purpose, etc.
E6 Now please think about your relationships with friends, family, and boyfriends or girlfriends. Not including horseplay or joking around, how many times in the past 3 months did… “someone threaten to hurt you, and you thought you might get really hurt?” “someone push, grab, shake, or choke you?” “someone hit you?” “someone beat you up?” “someone steal or destroy your property?” “someone scare you without laying a hand on you?”	Concern—this item asks about physical violence, a potentially triggering topic. Distal outcome—Pathways does not specifically target physical violence. We do not believe this item will be sensitive to the intervention. (See similar note above on D8)
E13 During the past 3 months, how many times were you in a physical fight?	Concern—this item asks about physical violence, a potentially triggering topic. Distal outcome— Pathways does not specifically target physical violence. We do not believe this item will be sensitive to the intervention. (See similar note above on D8)

Additionally, we propose moving Section K (Empowerment) so that it is between Sections D and E to help break up sections with more sensitive topics (Instrument 1).

It is important to us that youth feel safe and empowered taking the follow-up surveys. Furthermore, we feel that removing and rearranging these survey items will lessen emotional burden and secondary trauma, triggers, and sensitivities that have come up while completing the survey.²

Contact information requests

The current study includes periodic contact with youth asking them to contact the study team with any changes to contact information (Appendix C). We propose replacing the currently approved contact update process with four short contact information requests (Instrument 5). Consistent with the current timeline, the requests would be sent to youth 9 months, 15 months, 18 months, and 21 months after enrollment in the study. This population tends to be mobile with frequent changes to their contact information and can be difficult to locate. Additionally, they tend not have as many stable, consistent alternate contacts as other populations that we may typically contact to help locate respondents. For each update request (at 9-, 15-, 18-, and 21-months), we will contact the youth up to two times – an initial request and a reminder, if needed – using the most recent text and email contact information. The contact information request is shown in Instrument 5.

Given the importance of having updated contact information to achieve target response rates, we propose providing youth with a $5 digital gift card for each completed request. Furthermore, providing the digital token of appreciation immediately upon completion of the update may help to build confidence in the youth they will receive tokens of appreciate for all study activities, including contact updates. In line with a recent ASPE brief, providing a direct digital token of appreciation may prevent youth with lived experience from feeling undermined, disregarded, and/or marginalized.³ This request offers youth a quicker, more user-friendly way to provide that information at more frequent intervals.

The contact information update requests are estimated to take 5 minutes to complete (.08 hours) and that 50 percent of youth will complete each of the four requests.

Overall, the changes proposed in this request result in a decrease to the total annual burden estimates from 489 hours to 461 hours. Table 2 provides the updated burden estimates.

Table 2. Updated estimated annual burden with the removal of survey items and additional of contact information update

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total Burden (in hours)	Annual Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
SYSIL Youth Survey – Baseline	700	1	0.42	294	98	$12.32	$603.68
SYSIL Youth Survey – Follow-Up Survey 1 (6 months)	630	1	0.42	265	88	$12.32	$596.29
SYSIL Youth Survey – Follow-Up Survey 2 (12 months)	595	1	0.42	250	83	$12.32	$613.54
SYSIL Youth Survey – Follow-Up Survey 3 (24 months)4	490	1	0.42	206	69	$12.32	$595.06
Interview Guide for Pathways Sites (Treatment Sites)	30	1	1.5	45	15	$41.59	$623.85
Program Director Check-ins for Pathways Sites (Treatment Sites)*	10	2	.5	10	4	$41.59	$166.36
Interview Guide for Comparison Sites	30	1	1.5	45	15	$41.59	$623.85
Program Director Check-ins for Comparison Sites*	6	2	.5	6	2	$41.59	$83.18
Focus Group Discussion Guide for Pathways Youth (Treatment Youth)	50	1	1.5	75	25	$12.32	$154.00
Focus Group Discussion Guide for Comparison Youth	50	1	1.5	75	25	$12.32	$154.00
Contact Information Update Requests	350	4	.08	112	37	$12.32	$287.18
Total				1,382	461		$4,500.98

A revised Instrument 1, revised Appendix C, updated Supporting Statement A, Supporting Statement B, and Instrument 5 accompany this memo.

Time Sensitivities

We would appreciate approval as soon as possible since data collection is ongoing and we would like to make these adjustments for as many youth as possible as they complete the follow-up surveys.

1 Please note that survey items D6 and E5 are merely summaries of the instrument and do not include all survey items. The full revised survey can be found in the attached documents.

2 Office of the Assistant Secretary for Planning and Evaluation. “Report to Congress on Medicaid and

CHIP Methods and Emerging Strategies to Engage People with Lived Experience: Improving Federal Research, Policy, and Practice.” Fairfax, VA: December 20, 2021.

Available at https://aspe.hhs.gov/sites/default/files/documents/47f62cae96710d1fa13b0f590f2d1b03/lived-experience-brief.pdf.

3 Office of the Assistant Secretary for Planning and Evaluation. “Report to Congress on Medicaid and

CHIP Methods and Emerging Strategies to Engage People with Lived Experience: Improving Federal Research, Policy, and Practice.” Fairfax, VA: December 20, 2021.

Available at https://aspe.hhs.gov/sites/default/files/documents/47f62cae96710d1fa13b0f590f2d1b03/lived-experience-brief.pdf.

4 Data collection for the 24-month follow-up is likely to extend beyond 3 years. A request to continue data collection will be submitted prior to the expiration date of OMB approval.