Response to OMB Comments

Memo

To: Mary Mueggenborg

From: YARH Team

Date: 02/16/23, Revised 03/17/2023

Subject: Response to OMB comments

This memo provides responses to comments received verbally from OMB regarding a non-substantive change request submitted in December 2022 by OPRE to the approved information collection, Supporting Youth to be Successful in Life (SYSIL) Study (OMB #0970-0574).

Comment 1: I’m specifically interested in what evidence ACF has to support the rationale that this incentive level will be effective in encouraging youth to complete the contact information update.

Our current approach for collecting updated contact information from youth is to send them a text every 3 months asking that they contact us to provide any updated contact information. The amount of updated contact information received through this type of outreach has been minimal. We are reaching out to youth to complete the follow-up surveys and the lack of updated contact information presents a challenge for achieving target response rates for those surveys. We do not currently offer youth a token of appreciation for providing updated contact information. Literature has noted that providing an incentive increases response rates as compared to no incentive (Couper, 2008). The proposed $5 amount is considered in alignment with the effort of the request. We anticipate that providing a token of appreciation for youth to update their contact information will increase response to those requests, which will then increase the chances of locating youth for follow-up surveys, potentially boosting response rates for the follow-up surveys.    

Comment 2: In looking at Instrument 5, it appears that Mathematica will be collecting contact information every 3 months. However, the SSA shows 6-month intervals. Can you please clarify?

Table A.2 in SSA notes that the Contact Information Update Requests will be administered at 9, 15, 18 and 21 months post-baseline, while the introduction on Instrument 5 notes that contact update information requests will be sent out every 3 months. To clarify, starting at 9 months post-baseline, the contact information update requests will be sent every three months except when the timing intersects with another planned data collection (i.e., the 12 month survey). We are not planning to send the contact information requests at 3 months or 6 months because youth are likely to still be in frequent contact with their case workers and we can obtain updated contact information from the case workers and  the 6 month survey requests updated contact information. We have listed the timing of the survey data collection and contact information requests below (green font indicates the new survey and token of appreciation). We updated the introductory language for the contact information request (Instrument 5).   

Timeline for youth data collection and outreach (including tokens of appreciation):

0. Baseline survey - $40

1. 6 month follow-up survey - $45  

2. 9 month contact information request - $5

3. 12 month survey - $50

4. 15 month contact information request - $5

5. 18 month contact information request - $5

6. 21 month contact information request - $5

7. 24 month survey - $65

Proposed language update intro to Instrument 5 is as follows:

Thank you for your continued help with this important study! We will be getting in touch at various times every 3 months during the study to see how you are doing and get the latest information on how to reach you. This update should take you about 5 minutes to complete and you will receive a $5 e-gift card after you complete this update! The information you provide will be used to contact you about future SYSIL surveys to help us understand your experiences and will help shape programs and improve services provided to youth like you.

Comment 3 shared by OPRE: Our PRA coordinator, Andrew, wrapped up a meeting with OMB late yesterday [02/28/2023] and their Statistical Programs and Standards team had a recommendation to consider testing if an incentive at this level with this population would be effective using a split sample.

ACF and Mathematica appreciate the opportunity to learn from offering $5 electronic tokens of appreciation to youth and young adults to complete a short contact information update request. We discussed how to design an experiment with minimal disruption to the larger YARH-3 summative evaluation and equity for all participants.

We propose a randomized experiment to understand the effectiveness of offering a small token of appreciation (electronic gift card with value of $5) to improve contact information response rates. At the first contact information update request (9-months post-enrollment), youth would be randomly assigned to either be offered the $5 electronic gift card or no token of appreciation for completing the contact information request.  We would compare the response rates for that contact information response request for the two groups of youth (offered or not offered the $5 electronic gift card) to estimate the effect of the token of appreciation.

We would conduct this randomized experiment with all newly enrolled youth in all study locations (both intervention and comparison conditions).  To maintain equity for all youth, we would offer a total of $20 to all youth – the key difference being the amount received at the first and last request (see Table 1).  Notably, to offset the $0 token of appreciation at the 9-month contact information update for the control group, we will offer a $10 token of appreciation for completing the 21-month information update request to control youth.

Table 1. Tokens of appreciation by condition and contact request

Condition	9-month contact request	15-month contact request	18-month contact request	21-month contact request	Total
Intervention	$5	$5	$5	$5	$20
Control	$0	$5	$5	$10	$20

We look forward to OMB’s response to the proposed study. We are submitting an updated SSA, SSB, Instrument 5 (no changes since 02/2023), Appendix A, and Appendix C.

1100 First Street, NE, 12th Floor, Washington, DC 20002-4221 • (202) 484-9220 phone (202) 863-1763 fax • mathematica.org

An Affirmative Action/Equal Opportunity Employer