Personal Responsibility Education Program (PREP) Local Evaluation Support: PREP LES Annual Satisfaction Survey

OMB # 0970-0401

Expiration 6/30/2024

PREP LES Annual Satisfaction Survey

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN: The purpose of this information collection is to help state child welfare agencies develop more systematic data systems to track instability outcomes in children who exit foster care to adoption or guardianship. Public reporting burden for this collection of information is estimated to average 15 minutes per respondent, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0401 and the expiration date is 06/30/2024. If you have any comments on this collection of information, please contact Selma Caal, Selma.caal@acf.hhs.gov.

Introduction

As part of the Administration for Children and Families’ (ACF) Office of Planning, Research, and Evaluation’s (OPRE) PREP Local Evaluation Support and Dissemination (PLESD) project, we are looking to collect feedback about your experience receiving local evaluation support (LES) led by Abt Associates and partners. Specifically, we are conducting this survey to ask about your satisfaction with different PREP LES activities and resources.

Completing this voluntary survey should take no more than 15 minutes of your time. Your responses to these questions are anonymous and will not be shared with other PREP grantees. Individual responses will not be shared with OPRE or the Family and Youth Services Bureau (FYSB). Instead, the study team will combine responses across all respondents and by funding stream (i.e., PREIS, Tribal PREP, State PREP, Competitive PREP) to help assess and improve overall local evaluation support activities.

Part 1: About You

1. What type of PREP grant do you currently work on? (Select all that apply.)

   1. PREIS

   2. Tribal PREP

   3. State PREP

   4. Competitive PREP

2. Which best describes your role on the PREP grant?

   1. I’m part of the Implementation Team.

   2. I’m part of the Evaluation Team.

   3. Other: (Please describe)

3. In which ways have you engaged with PREP Local Evaluation Support (LES), if any? (Select all that apply.)

   1. My team meets regularly with a PREP LES liaison.

   2. I have attended a PREP LES webinar.

   3. I have attended a PREP LES Community of Learning (CoL) session.

   4. I have attended CPREP/SPREP office hours.

   5. I have used one or more resources provided by the PREP LES team.

   6. I have visited the PREP LES website (www.preplesd.com).

   7. Other: (Please describe)

   8. None of the above. (I have not engaged with PREP LES.)

Part 2: Satisfaction with PREP Local Evaluation Support (LES)

Thinking about your experiences with PREP Local Evaluation Support (LES) over the last year (including individual LES, webinars, Communities of Learning, and resources), please indicate the extent to which you agree or disagree with the following statements.

(IF YES to 3a): The regular LES calls with our liaison were beneficial to our evaluation.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(IF YES to 3a): The frequency of LES calls was sufficient to support our evaluation.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(IF YES to 3a): I felt comfortable asking our LES liaison questions.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(IF YES to 3a): Our LES liaison provided a timely response to my questions.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(IF YES to 3a): Our LES liaison was knowledgeable on the evaluation topic(s) we discussed.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(IF YES to 3a): Our LES liaison supported the development of our [evaluation plan/analysis plan/report].	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(For PREIS/TPREP:) The [Evaluation Plan/Analysis Plan/Reporting Template] was clear.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(For PREIS/TPREP:) The instructions for completing the [Evaluation Plan/Analysis Plan/Reporting Template] were clear.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(For PREIS/TPREP:) The [Evaluation Plan/Analysis Plan/Reporting Template] helped us write a strong, comprehensive [evaluation plan/analysis plan/report].	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(For PREIS/TPREP:) Adequate time was provided to complete the [Evaluation Plan/Analysis Plan/Reporting Template]	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3b): In general, the webinar(s) I attended was/were a good use of my time.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3b): I learned something at the webinar(s).	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3b): The knowledge and skills of the webinar presenters were appropriate.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3b): The webinar content was aligned with our evaluation goals.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3b): The knowledge and/or skills I acquired from the webinar were directly applicable to my work.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3c): In general, the Community of Learning (CoL) session(s) I attended supported our evaluation.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3c): I gained valuable insight from other evaluators through attending a Community of Learning (CoL) session.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3d): In general, the PREP LES Office Hours I attended supported our evaluation.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3e): The resources provided by the LES team were helpful for our evaluation.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3f): I found the PREP LES website to be user friendly.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3f): I was easily able to access resources and event information on the PREP LES website.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
(If YES to 3f): Materials I needed were available on the PREP LES website.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
The LES provided has been culturally relevant and respectful.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
The LES provided enhanced my understanding of conducting a rigorous evaluation.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
Overall, I believe our evaluation is of higher quality for having received LES.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A
Overall, the LES I received met my needs.	Strongly Disagree	Disagree	Neither agree nor disagree	Agree	Strongly agree	N/A

Part 3: Overall Feedback on Local Evaluation Support (LES)

1. What about the local evaluation support over the past year was most helpful? (Open-ended text box)

2. What about the local evaluation support over the past year was least helpful? (Open-ended text box)

3. How can we improve our local evaluation support in the future (including webinars, communities of learning, resources, and individual local evaluation support)? (Open-ended text box)

4. (If YES to 3h): Please share why you have not engaged with PREP Local Evaluation Support (LES).

Part 4: Suggestions for Local Evaluation Support (LES)

1. Please provide suggestions for webinars, communities of learning, tools, or resources that you believe would be beneficial for your evaluation. (Open-ended text box)

2. On which of the following topics would you be interested in receiving more LES? (Check all that apply)

1. Administrative/Secondary Data Collection

2. Attrition

3. Baseline Equivalence

4. Community Engagement in Evaluation

5. Conducting Random Assignment

6. Descriptive Study Design

7. Equity in Evaluation

8. Evaluation Recruitment and Retention

9. Fidelity of Implementation

10. Implementation Study Design

11. Instruments/Measures

12. IRB, Tribal Approvals, Research Applications, Consent/Assent

13. Logic Models

14. Quasi-Experimental Designs (QEDs)

15. Qualitative Analysis

16. Qualitative data collection (e.g., interviews, focus groups, observations)

17. Quantitative Analysis

18. Randomized Control Trial (RCT) Designs

19. Reporting and Dissemination

20. Research Questions

21. Sample Size/Power

22. Survey Data Collection

23. Other: (Please describe)

3. Please share any additional feedback, suggestions, or comments you have on the PREP Local Evaluation Support over the past year. (Open-ended text box)