Supporting Statement Part B FINAL

August 24, 2016

Maternal, Infant and Early Childhood Home Visiting Program:

Use of Cost Reporting Tool in Select Local Implementing Agencies

Supporting Statement

Part B

Contact:

Kyle Peplinski

Technical Monitor and Principal Investigator

Health Resources and Services Administration

Maternal and Child Health Bureau

Division of Home Visiting and Early Childhood Systems

5600 Fisher Lane, Rm. 10-86

Rockville, MD 20857

TABLE OF CONTENTS

Section Page

B. Collection of Information Employing Statistical Methods 1

B.1 Respondent Universe and Sampling Methods 1

B.2 Procedures for the Collection of Information 1

B.3 Methods to Maximize Response Rates and Deal with Nonresponse 4

B.4 Test of Procedures or Methods to be Undertaken 4

B.5 Individuals Consulted on Statistical Aspects and/or Analyzing Data 5

LIST OF ATTACHMENTS

Attachment 1. HVM-Cost Reporting Tool (HVM-CRT)

Attachment 2. Pilot Test Questionnaire

Attachment 3. Introductory/Invitation E-mail

Attachment 4. Reminder E-mail

Attachment 5. Thank You E-mail

Attachment 6. 60-day Federal Register Notice

Attachment 7. Comments Submitted in Response to 60-day Federal Register Notice

B. Collection of Information Employing Statistical Methods

B.1 Respondent Universe and Sampling Methods

The data collection process does not employ statistical methods. We plan to include approximately 90 local implementing agencies (LIAs) that receive funding from the Federal Home Visiting Program in the pilot test of the Home Visiting Model Cost Reporting Tool (HVM-CRT). LIAs are often community-based organizations, public health departments, or schools that implement national evidence-based home visiting models. The LIAs selected for participation in the pilot test will represent a subset of all Federal Home Visiting Program LIAs. We will work with the Health Resources and Services Administration (HRSA) to select 10 grantee states, with representation from the four main regions of the United States (Northeast, South, Midwest, and West). LIAs operating at least one of the nine evidence-based home visiting models currently implemented by the Federal Home Visiting Program grantees will be drawn from these states. We will work closely with national organizations that represent home visiting model developers and state grantees to identify the LIAs in these states to participate in the pilot test. Home visiting model developers are partnering with HRSA to support data collection from LIAs under a contract between HRSA and RTI and JBA. Home visiting model developers will develop a list of candidate LIAs. During a webinar, this list will be shared with HRSA and state grantees for review and feedback. The list will be revised based on this feedback. Because some of the national home visiting models are only implemented by a few LIAs, we expect to invite all of those models’ LIAs to participate, thus increasing the likelihood that we will obtain representation from all of the national home visiting models in the pilot test. After HRSA and state grantees agree on the LIAs to invite for participation, the LIAs will be contacted and their participation requested for the pilot test. Model developers will then provide the data collection tools to all LIAs that agree to participate.

B.2 Procedures for the Collection of Information

We have developed an Excel-based HVM-CRT to collect information from the LIAs (Attachment 1). Model developers will send the HVM-CRT to participating LIAs with a request to submit the completed form within 6 weeks. Prior to model developers’ sending the HVM-CRT to LIAs that agree to participate, we will conduct a 1-hour training webinar on data collection procedures and how to navigate the various sections of the HVM-CRT for the LIA participants. Detailed instructions are included in the HVM-CRT to assist the LIAs in providing the requested data accurately. All LIAs will also receive technical assistance on completing the Excel-based HVM-CRT. Because model developers have existing relationships with LIAs and LIAs are accustomed to providing data to them, they will be the primary point of contact for participating LIAs. However, if LIAs need technical assistance that model developers cannot provide, the RTI and JBA technical assistance team will answer LIA questions via phone calls, email, or webinars. Before data collection begins, we will provide an overview webinar for home visiting model developers and Federal Home Visiting Program grantees. In addition, we will provide ongoing technical assistance to these stakeholders to answer questions they may have or receive from LIAs.

We expect that model developers will send the HVM-CRT to respondents after approval from OMB is received in May 2017 and request that they complete and submit their forms via email to RTI within 6 weeks. We will ask model developers to follow up with participants within 2 weeks of sending the HVM-CRT to address questions and provide encouragement for them to submit the data by the requested due date. We will ask model developers to follow up again with respondents who have not yet submitted data during weeks 4, 5, and 6. If model developers are unable to make follow-up phone calls and send emails, RTI and JBA technical assistance team members will contact the LIAs on behalf of model developers.

Once the data are received, we will review them for accuracy and completeness. All data collected in the HVM-CRT will be assessed for missing information (i.e., percentage of fields with missing data) and incorrect data (i.e., percentage of data elements with formats that are not recognized; percentage with inappropriate range of values). We will also review whether the subcategories of costs sum to the expected total costs from total grantee subcontract budgets. We will contact model developers if we identify missing or unexpected values in their LIAs’ data submissions and ask them to work with the LIAs to submit accurate and complete data. Once all data collection is complete, model developers will send an evaluation questionnaire (Attachment 2) to all LIAs that participated to request their feedback on the HVM-CRT, including its content, usability, and any other suggestions for improvement. A flowchart depicting the HVM-CRT development and data collection process is presented in Exhibit B.2-1.

Exhibit B.2-1. Flowchart for Data Collection Process

Costs will be estimated for each of the LIAs after all data are submitted and cleaned. For these analyses, RTI and JBA will calculate costs for each cost category (e.g., model fees, salary and personnel, overhead costs, infrastructure, administrative supplies, service delivery materials, assessment, tools, and curricula, contract services, training, travel) and then sum across all cost categories to estimate total annual costs for each LIA and to examine the LIA costs for specific cost categories. We will use the LIA cost data to estimate mean, median, minimum, and maximum costs for the following measures:

• Annual cost per family served, by home visiting model

• For each home visiting model, distribution of average total annual cost per family served across key program components, such as number of home visit encounters

• LIA percentage cost allocations across the cost categories

This information will be summarized in a report that details these costs and describes the results of additional analyses examining the relationship between home visiting model or client characteristics on the costs per family.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

We assume that we will need to oversample LIAs in order to have representation from 90 LIAs across the nine national home visiting models. HRSA and the contractor team will identify approximately 10 Federal Home Visiting Program grantees for participation in the study, ensuring that the group of grantees asked to participate can provide representation for all nine national home visiting models included in the study. We will then work with home visiting model developers to determine which LIAs to invite to participate. HRSA and Federal Home Visiting Program grantees will then review and comment on the list of LIAs selected for participation. We do not expect that we will need to invite all LIAs in the 10-grantee states to participate. We expect that the 10 grantees invited to participate will have a total of approximately 250 LIAs. If 75% of these LIAs are willing to participate (n=187), and if 85% of the participants respond to the survey, we would end up with more than the desired number of LIAs (about 159 versus 90). For this reason, we anticipate asking model developers to select no more than 10 LIAs that implement their model in each state to limit the number of LIAs approached for participation in the 10 selected grantee states.

We further hope to maximize response rates by having home visiting model developers send an initial e-mail and then asking them to follow up to determine which LIAs are willing to participate. Model developers will follow up via e-mail with all invited participants within 2 weeks of the request and then follow up again with nonresponders at 4, 5, and 6 weeks. In addition, we will provide a webinar training for completing the HVM-CRT for participating LIAs, in collaboration with home visiting model developers and participating Federal Home Visiting Program grantees. Ongoing technical assistance will be available to model developers and their LIAs during the 6-week data collection effort to answer questions and encourage responses.

B.4 Test of Procedures or Methods to be Undertaken

Feedback from home visiting model developers was incorporated to create a final version of the Excel-based HVM-CRT. Findings from our analyses of the pilot cost data and from feedback that pilot LIAs provide about the HVM-CRT and its usability will be incorporated into recommendations for standardized cost reporting for the Federal Home Visiting Program and into a final, revised version of the HVM-CRT.

B.5 Individuals Consulted on Statistical Aspects and/or Analyzing Data

Kyle Peplinski, MA (301-443-7758), of HRSA is the Principal Investigator and Technical Monitor for the study. He has overall responsibility for overseeing the design and administration of the pilot cost reporting tool, and he will be responsible for analyzing the data.

RTI International is the project contractor responsible for developing the Excel-based version of the HVM-CRT; providing training, guidance, and technical assistance to the LIAs; and collecting and analyzing pilot cost data. Amanda Honeycutt, PhD (919-597-5129), serves as RTI’s Project Director. In this role, she is the primary contact with the Technical Monitor and oversees work on all project tasks.

The cost reporting tool, sampling and data collection procedures, and analysis plan were designed in collaboration with researchers at HRSA, RTI, and JBA. The following personnel were involved in the design of the protocol and data collection instrument:

Benjamin Yarnoff, PhD

Research Economist
RTI International

Specific Contribution: Cost Evaluation Model Task Leader

(919) 541-6640

Amanda Honeycutt, PhD
Senior Economist
RTI International
Specific Contribution: Project Director
(919) 597-5129

Jill Filene, MPH

Vice President of James Bell Associates

James Bell Associates

Specific Contribution: Senior Project Advisor

(954) 659-8677

Julie Leis, PhD

Senior Research Associate

James Bell Associates

Specific Contribution: Senior Research Analyst

(925) 523-3038