Appendix A - Synopsis of the PREIS Evaluation Plan Template and Guidance

Synopsis of the PREIS Evaluation Plan Template and Guidance, by Section

This two-page document presents a plain-language overview of the evaluation plan template for non-technical audiences.

1. Introduction

The evaluation plan introduction should summarize the rationale for the local evaluation and explain how the local evaluation will help to inform current and future programming and expand the evidence base on adolescent pregnancy prevention. In this section, evaluators (in collaboration with grantees) will describe the intervention, explain how it is intended to work, and describe who it is intended for.

2. Impact Evaluation

This section is the heart of the evaluation plan. After providing a brief high-level overview of the research design, evaluators will carefully specify the study’s research questions, describe the treatment and comparison conditions, and explain how study sites and participants will be recruited and retained. Evaluators must document all aspects of study intake and random assignment to ensure that treatment and comparisons groups can be meaningfully compared. Next, evaluators will specify their data collection plan: who will collect the data, how it will be collected, and the timing for baseline and follow up data collection. Evaluators will provide details on the study’s specific measures, tools, instruments, and necessary data sharing agreements; they will also outline potential challenges that could arise.

In this section, evaluators will also provide detail on specific elements of the evaluation such as the anticipated sample sizes, where the study will be registered, and whether there are any conflicts of interest.

Finally, evaluators will provide a brief overview of the analysis plan. The overview should include a power analysis to provide confidence that the study’s sample size will be large enough to produce meaningful results. Evaluators will develop more detailed analysis plans at a later time.

3. Process Evaluation (optional)

For studies that include an optional process or implementation study, evaluators will use this section to describe their process evaluation design plan. This section of the template is meant to provide guidance and suggestions for what topics to include in such a plan. No specific items are required.

Note that PREIS evaluations are not required to design or conduct a process/implementation study of their PREIS-funded interventions. Nonetheless, a process evaluation that provides high-quality implementation and performance feedback may yield important information about when, why, and how interventions work. A process study plan might include plans for defining and assessing fidelity of implementation; defining and measuring the reach of the intervention; and/or documenting implementation drivers, barriers to implementation, and any solutions to overcoming those barriers.

4. Other Evaluation Activities (optional)

This section is also optional. In it, evaluators can describe any other planned evaluation activities that are not part of the impact or (optional) process analyses. Examples might include descriptive analyses, a treatment-on-the-treated study linking dosage to outcomes, a mediational analysis, or an ethnographic study.

5. Approvals and Data Security

In this section, evaluators will describe their plans for obtaining IRB approval, providing staff trainings on human subjects protection, obtaining a federal-wide assurance, and protecting participants’ data and privacy.

6. Evaluation Plan Roles and Responsibilities

Here, evaluators will provide details on the evaluation team, and describe the role of each party.

Appendices

The two appendices provide optional tools that evaluators can use to think through certain topics and fill out corresponding sections of the template. These tools correspond to (A) logic model development and (B) specifying contrasts.