Planning Local Evaluations as part of the Personal Responsibility Education Program (PREP): Promising Youth Programs (PYP)

PREP-PYP Tribal Grantee Descriptive Evaluation MATHEMATICA POLICY RESEARCH

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this collection is 0970-0356; this number is valid through 03/31/2018.  Public reporting burden for this collection of information is estimated to average 300 minutes, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This collection of information is voluntary for individuals, but the information is required from Grantees.

Tribal prep Descriptive Evaluation template

Note: TPREP grantees would use this template if they plan to conduct a descriptive evaluation. TPREP grantees planning a rigorous impact evaluation will be asked to complete the impact evaluation template.

Instructions: This template is intended to gather pertinent details about your evaluation, including research questions, study design, program details, sample characteristics, data collection plans, and other details related to the feasibility of the study.

Please complete this form to the best of your ability given the status of the evaluation plan, fleshing out and updating the plans laid out in your application, as applicable. The written plan will be used internally between Grantee, local evaluator, project officer, and your TA liaison as the basis for discussion on phone calls during the planning period. These discussions will be used to further develop the plans and provide additional clarification as needed to finalize a feasible design that meets the grant and study objectives. Grantee plans must be approved by your FYSB project officer before proceeding with implementation.

Program Model

1. Name of Program:

2. Does the program incorporate an evidence-based program (with or without adaptations)? ___ Yes ____ No

If the program includes an evidence-based program model, please provide the name of model and briefly describe existing evidence or list references:

1. Do you plan to adapt the evidence-based model? ___ Yes ____ No

If yes, please describe the adaptations you plan to make to the model and the rationale for each adaptation you will make.

4. Is the program funded by this grant embedded as part of a larger set of services offered to tribal youth (e.g. funding a new program offering in an after school program)? ___ Yes ____ No

If yes, please describe how the funded program fits within the larger set of services.

5. Please complete the table below to describe all the components of the program funded under the Tribal PREP grant, including all Adulthood Preparation Subjects and any components of existing curricula being adapted/incorporated into the program.

Column A: List each component that will be offered (including any group or individual sessions, service referrals, service learning, or other services).

Column B: For each component, describe the amount, duration, and intended dosage (e.g. 5 sessions over 3 weeks for a total of 15 hours of programming).

Column C: Briefly describe the content of each component.

Example provided in first row of the table.

A: Component	B: Amount, duration, intended dosage	C: Content
Classroom lessons	5 sessions over 3 weeks for a total of 15 hours of programming	Lessons on contraceptive use and HIV prevention, decision making, and setting educational plans

6. For each component, please describe who will deliver or facilitate the component and the intended setting for offering the component. Example provided in first row of the table.

A: Component	B: Who will deliver?	C. Setting
Classroom lessons	Trained facilitators	After school program

7. Please provide a copy of the logic model for the program. The logic model should specify, at a minimum, inputs, activities, and expected outcomes for the program.

8. Please list what program components or adaptations are designed to meet the cultural and/or linguistic needs of the AI/AN tribal community

9. Please describe how the program will incorporate trauma-informed practices.

10. Please list any other services related to adolescent sexual health that are available to youth in the communities where your program will operate.

Evaluation Overview

11. Please list the research questions that will guide your descriptive evaluation. Please use concise language and frame these as questions rather than hypotheses.

1. Process evaluation questions (including how you will track activities and outcomes in your logic model, monitor fidelity and adaptations to the planned program model, and identify successes and challenges):

2. Outcome evaluation questions (including participant outcomes based on your logic model):

3. Other topics for descriptive evaluations, if applicable.

12. Please complete the following two tables.

In the first table, list all research questions described in question #11 in the column headings, then list all planned data sources in the rows under the data source heading. Then mark which data sources will be used to address which research questions.

Example provided in first row of each table.

Data Source	Research Question
	Did youth receive the intended dose of the program?
Attendance data	X

In the second table, identify all data sources, who will provide the data for each source, and when and how data collection for the source will occur.

Data Source	Respondent	Timing/periodicity of data collection	Data collection mode
Attendance data	Program staff	After each group session	Self-administered

Tribal Communities

13. Name of tribe(s) served by the program:

14. Please describe the size of the tribal communities included in the study, including an estimate of the number of youth eligible for the program.

15. Please describe the geographic region and dispersion (e.g. square miles of the region) of the tribal communities.

16. Please indicate your plans for incorporating input from tribal organizations and other partner agencies or community partners in the development of your evaluation plans and research questions.

17. Tribal communities typically require approval for research activities.

Do you have a clear understanding of the review processes for the community where you will be working? These may include a local IRB and/or tribal boards.

___ Yes ___ No

If yes, what is your estimated timeline for review – i.e., when you do you anticipate requesting approval from their entities, and how long do you expect the review process to take?

If no, please describe steps you will take to find out what tribal approval protocols are in place and what you will need to submit for review; specify a timeline for gathering this information and creating a clear plan for tribal review.

18. Does the tribal community restrict data sharing or ownership, require manuscript review and approval, or have other concerns of community confidentiality?

19. Are you aware of any laws specific to the tribal communities related to mandatory reporting of underage sexual activity or illegal substance use?

___ Yes I am aware, and laws exist ___ Yes I am aware, and no such laws exist

____ No, I am not aware if such laws exist

If you are aware such laws exist, please describe your plans to accommodate them in your study protocols.

If you are unsure whether such laws exist, please describe steps you will take to find out if they exist in the communities involved in the study.

Youth Target Population and How They Will Be Enrolled and Retained

20. Please describe the characteristics of the youth population you plan to serve with the program.

Age:

Grade level:

Race/ethnicity or tribe:

Gender:

Risk characteristics:

Other characteristics:

21. What, if any, eligibility criteria will be used to identify youth for the program?

22. Please describe the criteria you will use to select your evaluation sample and how the characteristics of the evaluation sample will or will not differ from program participants not included in the evaluation.

23. Please provide the expected start and end dates for program and evaluation enrollment.

	Program Enrollment	Evaluation Enrollment
Expected start date
Expected end date

24. Please provide the number of cohorts, sites, and youth you anticipate will be enrolled in the program and evaluation.

	Program Enrollment	Evaluation Enrollment
Number of cohorts
Number of sites in each cohort
Number of youth per cohort

Note: if you have a program that enrolls continuously, please just provide the number of sites and youth.

25. If the number of expected youth to be enrolled varies between the program and evaluation, please describe why the enrollment levels vary.

26. Please describe your planned strategies, including drawing on community strengths, to recruit youth into the study.

27. Please describe any anticipated challenges related to reaching the intended AI/AN youth population.

28. Please describe how the recruitment strategies will address the recruitment challenges.

29. Please describe the procedures you will use to collect consent from study participants, or consent from their parent or guardian and assent from the participant (if needed).

30. Please describe any exceptions to obtaining parental consent in the communities involved in the evaluation.

31. Please describe how you will engage youth to begin the program and retain them to participate in the program’s services over time.

32. Please describe the strategies you will use to track and retain youth enrolled in the study.

Outcomes, Data Collection, and Planned Analysis

33. Please fill in the table below for each of piece of data you will collect to measure participant outcomes.

Example provided in first row of the table.

Participant outcome	Measure	Timing of data collection	Which data source will include this measure	Who will be responsible for data collection
Sexual initiation/activity (vaginal)	Ever engaged in sex	Program enrollment, 3 months post-program	Youth survey	Program staff administer during initial group session, evaluator contact youth 3 months post-program for web-based survey

34. Please fill in the table below for each of piece of data you will collect to measure the process of service delivery. Examples of process measures are attendance, reach, dosage, and fidelity to the program model.

Example provided in first row of the table.

Process outcome	Measure	Which data source will include this measure
Youth participation	Whether youth attended each group session	Attendance data

35. Please briefly describe your plans for analyzing the process and outcome data and how the data will help to answer your research questions.

DRAFT /home/ec2-user/sec/disk/omb/icr/201702-0970-001/doc/71285601 15 01/23/21 0:21 A1/P1