Multi-Site Implementation Evaluation of Tribal Home Visiting
OMB Information Collection Request
New Collection
Supporting Statement
Part A: Justification
Non-Substantive Change Request February 2019
Initial Request Approved December 2018
Submitted July 2018
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
330 C Street, S.W.
Washington, D.C. 20201
Project Officer:
Aleta Meyer
Information Collection Request Summary 1
A1. Necessity for the Data Collection 2
A2. Purpose of Survey and Data Collection Procedures 4
A3. Improved Information Technology to Reduce Burden 10
A4. Efforts to Identify Duplication 11
A.5 Involvement of Small Organizations 11
A6. Consequences of Less Frequent Data Collection 11
A8. Federal Register Notice and Consultation 12
A9. Incentives for Respondents 13
A10. Privacy of Respondents 15
A12. Estimation of Information Collection Burden 17
A13. Cost Burden to Respondents or Record Keepers 20
A14. Estimate of Cost to the Federal Government 20
A16. Plan and Time Schedule for Information Collection, Tabulation and Publication 21
A17. Reasons Not to Display OMB Expiration Date 23
A18. Exceptions to Certification for Paperwork Reduction Act Submissions 23
ATTACHMENTS:
Attachment A. Informed Consent Forms
Attachment B. 60 Day Federal Register Notice
Attachment C. Agreement to Keep Participant Data Private
Attachment D. Documentation of Initial IRB Approval and Amendment Approval for Waiver of Parental Consent for Minor Caregivers (Ages 14-17)
Attachment K. MIECHV Strategic Learning Agenda
List of Exhibits
Exhibit A.1. Data Sources by Evaluation Aims and Questions 9
Exhibit A.3 MUSE Information Collection Burden Table 19
Exhibit A.4 MUSE Study Time Schedule 21
Exhibit A.5 MUSE Study Data Collection Schedule by Instrument 22
Status of study:
This is a new information collection as part of the Multi-Site Implementation Evaluation of Tribal Home Visiting study.
What is being evaluated:
Multi-site evaluation of federally funded Tribal Maternal, Infant and Early Childhood Home Visiting (MIECHV) programs.
Type of study:
Implementation evaluation using a mixed-methods approach.
Utility of the information collection:
This study is the first multi-site, multi-model study that will systematically explore how home visiting programs are operating across diverse tribal community contexts and identify factors that lead to implementation successes. The study will address a gap in the evidence base regarding the provision of home visiting services in tribal and urban Indian communities.
MUSE will provide information that will help the federal government design and support federal home visiting initiatives in tribal communities and similar populations.
Tribal MIECHV programs will have access to study findings for use in decision making about improving home visiting services for children and families.
The information generated in this study is also expected to be used by technical assistance providers, the scientific community, and other aligned professionals for the purposes of refining technical assistance and other program supports and prioritizing future research agendas.
The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval for a study of Tribal Maternal, Infant and Early Childhood Home Visiting (MIECHV) programs. The Multi-Site Implementation Evaluation of Tribal Home Visiting (MUSE) is the first multi-site, multi-model study that will systematically explore how home visiting programs are operating across diverse tribal community contexts and identify factors that lead to successful program implementation. The evaluation will provide information that will help the federal government design and support federal home visiting initiatives in tribal communities and similar populations. Evaluation findings will also assist programs with improving home visiting services for children and families.
Through the proposed information collection, the evaluators will obtain information about the planning processes employed by Tribal MIECHV programs, the services implemented by those programs, the characteristics of staff and caregivers receiving services, and how staff and caregivers experience home visiting services. The evaluation will collect information through secondary data analysis, surveys of program staff and caregivers, qualitative interviews with staff and caregivers and administrative program data.
The MIECHV program has been engaged in a broad portfolio of research, evaluation, and performance measurement since its inception in 2010. The evidence generated by each of these activities contributes unique perspectives to the understanding of the MIECHV program and to the overall MIECHV Learning Agenda (see Attachment K). The MUSE study builds on lessons learned from the Maternal and Infant Home Visiting Program Evaluation (MIHOPE) Implementation Study. The study’s recent report1 (Duggan et al., 2018) is similarly focused on addressing knowledge gaps related to implementation of home visiting programs. In designing the MUSE study, the research team consulted with the MIHOPE team to create a design and measurement strategy that represents the best of current approaches for studying implementation. These consultations allowed for more efficient and effective data collection strategies and tools to be integrated into the MUSE study design. Additionally, with a view towards a potential future study of the impact of the Tribal MIECHV program as part of the MIECHV Learning Agenda, the MUSE team is including measures of positive caregiver outcomes, including parenting self-efficacy and social support, to gain an understanding of how those measures operate with tribal populations.
Examining a Distinctive Federal Approach to a Tribal Grant Program
MUSE is designed to provide useful information for the Tribal MIECHV program office to ensure that the goals of building capacity to serve American Indian/Alaska Native (AIAN) families are being met, to tailor program guidance, and increase the usefulness of support provided to grantees. As part of the MIECHV Learning Agenda, this study is designed to generate findings that can inform future federal investments by ACF. Questions exist around the use of intervention models that were created and tested with the general population in tribal communities, whether these models can be implemented as designed in different communities, and whether intentional support for capacity building is necessary for translating evidence-based models into different community contexts.
Through local, grantee-designed and -led single-site rigorous evaluations conducted between 2012 and 2016, Tribal MIECHV grantees assessed the effectiveness of their home visiting programs across a variety of child and parent outcomes (Roberts et al, 2018). These grantee evaluations conducted during the first Tribal MIECHV grant cycle addressed research questions that reflected local community priorities and yielded important findings about individual home visiting programs, but the applicability of these findings beyond the local contexts in which they occurred is somewhat limited. More research is needed that examines key questions across diverse tribal home visiting contexts. Small sample sizes, family recruitment and attrition, and staff turnover limited some grantees’ abilities to draw conclusions about home visiting in their communities. A multi-site study is a unique opportunity to pool data from tribal communities with small samples sizes to answer important research questions. A multi-site study can systematically address challenges related to family recruitment and attrition and staff turnover that complicated previous evaluation efforts. Additionally, most grantee evaluations focused on understanding whether their local programs achieved child and parent outcomes; local evaluations did not examine implementation. More research on implementation is needed to interpret findings from outcome studies in Tribal MIECHV communities. Unlike these prior studies that primarily focused on outcomes, MUSE is designed to answer key questions about implementation of home visiting across diverse tribal contexts. This purpose frames the scope of the MUSE study and what it will and will not tell us about tribal home visiting. It will provide information about what kinds of families are getting home visits, how programs are structured to provide those visits, how staff are supported to be effective home visitors, and what happens within visits. It will not test the effectiveness of home visiting models in changing family outcomes.
Informing Tribal Communities’ Efforts to Improve Services to Families
Grantees have repeatedly expressed their desire that the proposed study generate findings that can help them better serve children and families in their communities. The MUSE study offers grantees, specifically those with relatively small participant numbers, the opportunity to generate findings that are both scientifically robust and meaningful to tribal settings and members. Our community-engaged process for developing the study design ensures that the findings from MUSE can be applied by tribal programs throughout the country, given that a primary goal of MUSE is to explore supports and challenges to home visiting implementation specifically in tribal communities. The mixed-method multi-site study design has the potential to generate new knowledge about family recruitment and engagement; staffing; training, supervision and support for staff; and processes for developing cultural modifications.
Legal or Administrative Requirements that Necessitate the Collection
There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.
Overview of Purpose and Approach
MUSE is a multi-site evaluation research study of the implementation of Tribal MIECHV programs that has been designed and will be conducted within a participatory evaluation framework. MUSE uses a mixed-method evaluation design to better understand home visiting implementation across the Tribal MIECHV initiative from the perspectives of caregivers receiving services and staff. The study incorporates two types of descriptive research as delineated by the ACF Common Framework for Research and Evaluation2, foundational and exploratory. As foundational research, MUSE seeks to comprehensively describe interventions, services, programs, and policies being implemented across grantees, identify influential conceptual frameworks and local theories of change, and characterize caregivers and their experience with services. As exploratory research, the study will conduct preliminary examinations of relationships between core constructs, drawn from human services and implementation science, to generate evidence of potential connections between program inputs and outputs.
MUSE will use blended concurrent and sequential collection of qualitative and quantitative data to optimize our ability to identify and respond to emergent findings in a pragmatic and feasible way. The study approach will include secondary data analysis, qualitative interviews, administrative data, longitudinal surveys of caregivers, home visit questionnaires, staff surveys, and logs of program activities.
Grantees will have the option to make requests for data specific to their program to use for program planning, grant funding, or other similar purposes, which the MUSE study team will accommodate when capacity allows and when data reporting numbers are sufficient to protect participant privacy. Representatives from participating grantees will be part of a dissemination committee, providing input on dissemination priorities; collaborating on analysis, interpretation and dissemination of findings; and approving all final dissemination materials. Participating grantees will be made aware of and receive interim and final reports, highlighting the findings and recommendations most relevant for this target audience.
Research Questions
The evaluation questions are organized around three primary study aims:
Aim 1: Identify and describe primary influences shaping tribal home visiting program planning.
Aim 2: Identify and describe how tribal home visiting (THV) programs are being implemented.
Aim 3: Explore supports and challenges to home visiting implementation in tribal communities.
Through Aim 1, the evaluation team will describe the tribal home visiting programs as planned by grantees and examine the processes that influenced the design of local programs. The following questions will be examined under Aim 1:
What is the local context that informs planning?
How are implementation science principles reflected in local program planning?
What is the degree of alignment between local context and implementation science principles? How are differences addressed in planning?
How do model requirements inform program planning?
What adaptations, enhancements, and supplements to existing home visiting models are planned? Why?
Through Aim 2, the evaluation team will describe the actual implementation of programs. The following questions will be addressed through Aim 2:
How are THV programs staffed and what are the characteristics of those staff?
What services are provided to families?
What are caregivers’ experiences with services?
What happens during home visits?
What training, support and supervision do staff receive?
What are the characteristics of families served and do those characteristics change over time?
Under Aim 3, the evaluation will explore associations between elements of THV to identify influencing factors that both support and challenge program implementation. The fifth question under Aim 3 will allow us to explore the relationship between program planning, addressed in Aim 1, and program implementation, addressed in Aim 2. The following evaluation questions will be addressed under Aim 3:
What influences relationships between home visitors and families?
What influences what happens during home visits?
What influences the amount of home visiting families receive?
What influences staff self-efficacy and job satisfaction?
What influences tribal home visiting programs’ ability to implement their programs as intended?
Study Design
MUSE addresses the aims in the section above using a rigorous, mixed methods approach. Mixed-methods allows the MUSE Team to integrate multiple theoretical approaches and worldviews into the study design and application. This method increases study relevance, reduces limitations of strictly qualitative (e.g. reduced generalizability) or quantitative studies (e.g. limitations associated with statistical power) and enhances our ability to pursue a study that is responsive to stakeholders and has a high level of cultural rigor. The theoretical orientation of this study is informed by multiple domains including child development, implementation science, and community-engaged research.
Quantitative and qualitative data will be collected over approximately two years. The use of multiple data sources will ensure that constructs are queried at various time points in implementation and with different modalities (i.e. quick reflection on a particular home visit and more global feedback across months of service provision). MUSE will use a blended concurrent and sequential collection of qualitative and quantitative data to optimize our ability to identify and respond to emergent findings in a pragmatic and feasible way. This approach will enable strategic expansion, triangulation and elaboration of data, thereby ensuring rich, dependable and highly contextualized findings. Sequential, mixed method data collection will enable investigators to not only pursue the aims separately, but also to elucidate relationships across aims. For example, the research activities addressing Aim 1 will serve as a foundation with which findings from Aim 2 activities can be interpreted and analyzed. Such analysis will generate important results clarifying the way in which planning processes at multiple levels (e.g. initiative planning by funder and local implementation planning) influence actual implementation. Understanding the dynamics between planning and implementation processes has the potential to inform practices for multiple stakeholders including funders, model developers, and program implementers.
In addition to data collection, points of interface between qualitative and quantitative data will occur during data analysis and interpretation, further enabling triangulation and comparison of data across sources. This process will require strategic integration of data in dynamic data sets and could include techniques associated with data merging, connecting and/or embedding depending on the particular relationship or question.
Universe of Data Collection Efforts
The following is a summary of the data collection methods and instruments that will be used to answer the MUSE research questions (RQ). See Exhibit A.1 for a matrix of research questions by data sources.
Secondary Data Analysis (RQ 1, 2, 3, 4, 5, 16)
Systematic review of 17 grantee implementation plans. These plans are existing documents for which burden was approved through clearance process PRA#0970-0389 (exp. 8/31/2019). Therefore, there are no instruments or burden included for this effort within this current package.
Caregiver Enrollment Form (RQ 7, 8, 9, 11, 12, 13, 14, 15, 16) [Instrument 1]
Home Visitors will enter basic information about caregivers into the Caregiver Enrollment Form after they have consented to participate in MUSE.
Data from this form will be used to track data collection on caregivers who have agreed to participate in MUSE, and to manage and ensure the quality of the data.
Caregiver Surveys (RQ 7, 8, 9, 11, 12, 13, 14, 16) [Instruments 2 & 3]
Caregivers will be asked to complete a survey at baseline, 6 months, and 12 months after study enrollment.
The Caregiver Surveys will collect information on four key constructs: social support, parenting emotion regulation, parenting self-efficacy and communal-efficacy, and caregivers’ experiences with home visiting.
Social support will be measured using an abbreviated version of the Social Provisions Scale, plus 2 additional items added by MUSE to measure specific support around parenting.
Parenting emotion regulation will be measured using items from the Mindful Attention and Awareness Scale, Interpersonal Mindfulness in Parenting Scale, and Mindful Teaching Scale. The MUSE Team has adapted items from these scales to be more appropriate for parents of young children and have added two items specifically to assess parents’ ability to set aside their own stressors to attend to their children.
Parenting self-efficacy will be measured using an adapted version of the 7-item Parenting Self-Efficacy subscale from the Parenting Sense of Competence scale. A measure of Parenting Communal-Efficacy created by the MUSE Team will also be used to assess the extent to which caregivers’ parenting efficacy is derived in part from the supports of family and community.
Caregivers’ experiences with home visiting will be measured using questions created by the MUSE Team to help us better understand caregivers’ relationships with their home visitors, their expectations of the program, and whether those expectations have been met. The survey also includes questions adapted from a survey developed by the Home Visiting Applied Research Collaborative.
Rapid Reflect Self-Completed Home Visit Questionnaires (RQ 7, 8, 9, 12, 13) [Instruments 4 & 5]
The MUSE Team has developed self-completed questionnaires for home visitors and for caregivers.
The Caregiver Rapid Reflect asks about caregivers’ satisfaction with the home visit and their home visitor.
The Home Visitor Rapid Reflect collects information on travel time to the home visit, length and location of the home visit, people participating in the home visit, the content covered in a home visit, the responsiveness of the home visitor to emerging needs and interests, challenges encountered by the home visitor during the visit, and engagement of the participating caregiver.
Staff Surveys (RQ 6, 7, 10, 15) [Instruments 6, 7, 8, & 9]
All program directors, program managers/coordinators, home visitors, and local program evaluators will be asked to complete a one-time staff survey on their experience with the tribal home visiting program. If a staff person plays more than one role within the program, they will only receive one survey.
The staff surveys contain some common domains across roles that will allow us to explore similarities and differences across role types (e.g., professional background, role within the program, organizational culture and climate, job satisfaction, professional quality of life, perceived program effectiveness, connection to community served and demographics) but most survey domains are tailored by role.
Program directors will be asked about prioritized outcomes, the program’s ability to prepare home visitors, expectations of home visitors, self-efficacy in the program director role, leadership style, and perceptions of training received related to the program director role.
Program managers/coordinators will be asked about alignment with model prioritized outcomes, goal setting, expectations of home visitors, self-efficacy in program management and supervisory role, leadership style, perceptions of training received related to program management and supervision, and experience providing supervision and home visit observations.
Home visitors will be asked about the service environment, home visit planning and resources, perceptions of the home visitor role, self-efficacy in the home visitor role, perceptions of home visitor training, supervision and peer support received, and reflective supervision.
Local program evaluators will be asked about the importance of evaluation-related skills, involvement with the program, amount of interaction with program staff, data systems, performance measurement. CQI and role satisfaction.
Program Implementation Survey (RQ 6 & 7) [Instrument 10]
Program managers/coordinators will be asked to complete a short survey on program implementation activities. This survey can be completed jointly with other managers at the grantee site if needed.
This survey asks about services available in the community, sources of program funding, program eligibility criteria, use of program incentives, staffing levels, caseloads, and home visitor recruitment and hiring.
Qualitative Interviews (RQ 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16) [Instruments 11, 12, 13, & 14]
Interviews will be conducted by study personnel during grantee site visits with individuals in the five primary roles: program directors, program coordinators/managers, home visitors, local program evaluators, and caregivers participating in home visiting.
Qualitative interviews will be used to elucidate four primary areas: 1) real-world implementation, including the impact of COVID-19 and related responses on implementation; 2) home visitor support and supervision; 3) home visitor-family relationship; and 4) the home visit.
The interviews will follow a semi-structured format allowing for some questions to be asked consistently across grantees while also enabling grantee- and interviewee-specific questions and question tailoring when necessary.
The program director and program coordinator/manager interview protocols include questions about staffing; services provided; program planning; model selection; model fit; adaptations, enhancements and supplements; technical assistance; home visitor support and supervision; why families enroll; successful parenting; program impact; program manager role; role of community in program decisions; and the impact of COVID-19 and related responses on home visiting.
The home visitor interview protocol includes questions about home visitor support and supervision, model fit, why families enroll, successful parenting, family experiences, perceived program effectiveness, relationships with caregivers, what happens during a typical home visit, a description of a great home visit; and the impact of COVID-19 and related responses on home visiting.
The local program evaluator interview protocol includes questions about the role of the evaluator, supports available, challenges experienced, data systems, data use, interactions with program staff, model selection, program planning, technical assistance, the perceived impact of evaluation within the organization, and the impact of COVID-19 and related responses on home visiting.
The caregivers interview protocol includes questions about their expectations for home visiting, home visitor-family relationship, what happens during a typical home visit, description of a great home visit, screenings, visit preferences, support from home visitor, skills gained, changes and benefits experienced, impact of COVID-19 and related responses on home visits and parenting.
Implementation Logs (RQ 6, 7, 10) [Instrument 15]
Program managers/coordinators will be asked to submit Implementation Log data each month.
Implementation Logs will cover information about staff changes, training, individual and group supervision, and family group activities.
Administrative Program Data (RQ 7, 11, 12, 13, 14) [Instrument 16]
Grantees will be asked to designate a staff person(s) to submit Administrative Program Data to MUSE
The MUSE study will utilize existing individual-level data submitted by caregivers to their local home visiting programs.
The MUSE study will collect four types of Administrative Program Data: caregiver demographics, screener data, home visit participation data, and group activity participation data.
Grantees will have the option to submit item-level substance use screener data and summary score depression screener data.
The MUSE Team will ask grantees to submit locally collected process data on the services provided to families, most of which is already collected for home visiting model developers or federal reporting requirements. These data include clients’ date of referral and referral source, length of participation in THV, number and frequency of home visits, visit mode, and participation in family group events.
Evaluation Questions |
Data Sources |
||||||
Qualitative |
Quantitative |
||||||
Secondary Data Analysis of Implementation Plans |
In-Person Qualitative Interviews |
Administrative Program Data |
Caregiver Surveys |
Rapid Reflect Questionnaires |
Staff Surveys & Program Implementation Survey |
Implementation Logs |
|
Aim 1. Identify and describe primary influences shaping tribal home visiting program planning |
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Aim 2. Identify and describe how tribal home visiting programs are being implemented |
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Aim 3: Explore supports and challenges to home visiting implementation in tribal communities |
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
MUSE data collection will occur for approximately 27 months. Once OMB approval is obtained, participating grantees will begin enrolling caregivers into MUSE and collecting Caregiver Surveys, and the Rapid Reflect Self-Completed Home Visit Questionnaire. These two caregiver-focused data collection activities will occur throughout the MUSE data collection period. The staff surveys will be administered once, within the first month of the study. New staff who begin working after the initial staff survey is conducted will be surveyed on a rolling basis throughout the length of the study. Qualitative interviews will be conducted virtually or during one site visit made to each participating grantee. These interviews will occur throughout the data collection period. The Implementation Logs will be completed monthly for two years. Participating grantees will submit administrative data every six months for a total of four times. Because administrative data will include data on service delivery for the two-year data collection period, the final administrative data submission will occur approximately 27 months after the data collection period begins.
Respondents will complete all surveys and logs using a computer, phone, or handheld device which requires a lower time commitment compared to paper and pencil surveys. Caregivers receiving in-person home visiting will use a touch screen to complete surveys, which further reduces response time and is more intuitive than using a keyboard. Caregivers receiving virtual home visiting can take the survey over the phone or using a web survey link on a personal computer or device Reports and reminders generated by the MUSE Team will be made available to home visiting staff to reference data collection schedules for participating caregivers and keep track of surveys that need to be completed. Administrative data will be submitted through a secure cloud-based portal with drag and drop capabilities. This mechanism for submitting data entails lower burden than emailing files or sending them through secured mail. For the qualitative interviews, the data collection team will travel to the home visiting program office, or conduct the interviews virtually, to reduce the travel burden on staff participating in interviews. The interviews will be audio recorded with participant consent (see Attachment A).
The data requirements for this study have been carefully reviewed to determine whether the needed information is already available. Efforts to identify duplication included a review of the current literature and discussions with knowledgeable experts. Limited information is available about home visiting implementation in tribal communities, and the few existing studies focus on a single home visiting model. This study will be the first multi-site, multi-model study of home visiting implementation in tribal communities. No existing data source can provide the data needed to answer the study’s research questions.
MUSE will not generate data that is duplicative of information accessible to ACF; rather, the study will analyze data that are already collected by grantees to satisfy various reporting requirements. THV grantees are required to calculate and report aggregate demographic, service utilization and performance measurement data to ACF annually. MUSE will also utilize existing information by conducting a content analysis of grantee implementation plans and performance measurement data submitted to ACF.
Implementation plans and collection of demographic and service utilization data are covered under OMB control number 0970-0389, expiration date 8/31/2019.3
Collection of performance measurement data are covered under OMB control number 0970-0500, expiration date 8/31/2020.4 Home visiting models require programs to collect and report data on caregivers and the services they receive.
Some of the organizations involved in this study are small, non-profit organizations. The research team has minimized burden by keeping the interviews and surveys as short as possible, only asking about information directly tied to the study’s aims and questions, and scheduling interviews on-site and at times convenient for the respondents.
When developing the design for the MUSE study, we carefully considered the respondent burden associated with different research questions and methodologies. We elected to conduct staff surveys once and assess staff characteristics at a single point in time rather than observe change over time. Similarly, we opted to conduct one round of qualitative interviews instead of multiple rounds of interviews to limit burden on staff and caregivers.
Caregiver Surveys are collected three times: at baseline, six months and 12 months. Other studies have shown that twelve-month attrition rates can reach 50% or higher (Gomby, et al., 1999; O’Brien, et. al., 2012), so including the six-month time point will allow us to gather follow-up data from more respondents than if we only included a twelve-month follow-up, thereby increasing our sample size and power to estimate effects. Less frequent data collection would result in fewer paired baseline and follow-up surveys.
To reduce the burden on home visitors and caregivers, we have opted to collect data on what happens during individual home visits (Rapid Reflect) for a twenty-two-month period on a sample of visits instead of all home visits conducted during the data collection period. The Rapid Reflect will be completed for each home visit completed by home visitors at the grantee organization one week out of each month. This is an effective sampling rate of 23% of all home visits conducted. Reducing the sampling of home visits further, by collecting the Rapid Reflect less frequently, would limit the variation and representativeness of data across caregivers and home visits, and limit the power to detect effects in statistical analyses.
There are no special circumstances for this data collection.
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13 and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995)), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on February 28, 2018, Volume 83, Number 40, pages 8681-8682, and provided a 60-day period for public comment. This notice included information about all possible burden under this OMB number and the future submissions for anything described but not yet submitted as final will be announced in a 30-day Federal Register Notice only. A copy of the 60-day notice is included as Attachment B. During the notice and comment period, three comments and two requests for the instruments were received. Changes to the instruments were made in response to public comments received, additional feedback provided by MUSE stakeholders, and needed refinements identified as the MUSE Team further developed the detailed data collection protocols and technologies.
The MUSE Team conducted telephone consultations with experts in the fields of home visiting, implementation science, and evaluation of tribal programs. The Team held multiple consultations with researchers that conducted the MIHOPE implementation study. We consulted with these experts on study design, measurement constructs, instrument development, sampling and potential analyses. The MUSE Team also convened a Technical Workgroup (TWG) made up of a subset of the aforementioned experts as well as Tribal MIECHV grantee staff, Federal Tribal MIECHV staff, and Tribal MIECHV technical assistance providers. Expert consultants and the TWG provided consultation on the conceptual model, proposed study design, preliminary analysis plans, data collection instruments and working with grantees to collect program and caregiver data. Their recommendations helped shape the final study design. The MUSE Team will continue to convene the TWG throughout the MUSE study.
The MUSE study places burden on caregivers receiving home visiting services by asking them to participate in repeated data collection. A subset of caregivers participating in MUSE (caregivers who enroll in home visiting after MUSE begins) will be invited to take the Caregiver Survey three times and retaining these respondents over time is integral to the quality of the data collection. We will ask these caregivers to take a 15-minute baseline survey, a 30-minute survey at six months, and a 30-minute survey again at 12-months. Home visiting staff will be trained in protocols to ensure that caregivers are given privacy to complete surveys, and all staff collecting these data will be certified in human subjects’ research protections. In addition, surveys will be collected on tablets using secure software that ensures caregiver responses cannot be accessed by home visiting program staff (see Supporting Statement B, section B2 for additional information). We will offer caregivers a gift card at each time point: $10 at baseline, $15 at six months, and $15 at 12 months. We will conduct qualitative interviews with approximately three caregivers at each grantee. These interviews will take approximately 1 hour. We will offer a $40 gift card to caregivers who participate in an interview. Exhibit A.2 provides an overview of the incentives to be provided.
Exhibit A.2 Proposed Incentives for Caregivers Participating in Caregiver Surveys and Qualitative Interviews |
||
Data Collection Method |
Incentive Amount |
Estimated Time to Complete |
Caregiver Surveys |
|
|
Baseline (at enrollment) |
$10 |
15 minutes |
6-month follow-up |
$15 |
30 minutes |
12-month follow-up |
$15 |
30 minutes |
Qualitative Interviews of Caregivers |
|
|
In-person interview |
$40 |
1 hour |
TOTAL |
$80 |
|
There are two primary reasons for providing incentives to caregivers participating in the MUSE study and both are critical to the scientific integrity of the study: respecting cultural protocols in the communities participating in this study and reducing non-response bias by obtaining and maintaining a representative sample of caregivers over time.
Demonstrating Respect for Cultural Protocols
We are keenly aware of the specific cultural and contextual importance of incentives in research, given our experience working with tribal communities. In previous studies, partners, including tribal MIECHV grantees and tribal communities, conveyed the importance of providing incentives. In our team’s experience within tribal contexts, the use of incentives for participation in research is expected by both research participants and tribal leaders. This is especially true for studies conducted by researchers external to the community.
Best practice guidelines developed by AIAN communities confirm our experience, explicitly stating that incentivizing respondents for their time is an essential element of reciprocity between the researcher and AIAN knowledge holders (Mihesuah 1993; Nielsen et al., 2007; Sobeck et al., 2003; Davis, 1999). The National Congress of American Indians includes incentives as an important component of community based participatory research (Sahota, 2010). Tribal research review entities such as Tribal IRBs and Tribal Councils have codified these community norms and best practices by requiring compensation for research participants as a condition of study approval (Doughty, 2017).
Incentives reinforce the notion that the knowledge being shared by local participants is valued, respected, and honored. Mutual respect and reciprocity are strong cultural traditions in many AIAN cultures. Offering incentives to study participants is a way of offering recognition of the value of the knowledge being shared with the study team and establishing trust. This is particularly critical given the context of research in tribal communities and past histories of abuses in which researchers went into communities, took away data and local knowledge, and offered nothing in return (Pacheco, et al., 2013). Offering an incentive at the outset is an important step in establishing trust, making research culturally valid and ultimately obtaining scientifically rigorous data (Tribal Evaluation Workgroup, 2013). Incentives demonstrate respect for individual community members’ contribution to research and attempt to ameliorate logistical challenges associated with participating in research in many tribal communities (Doughty, 2017). In each of the 17 communities that intend to participate in MUSE, the appropriate research review entity for that grantee must review and approve the MUSE research protocol in order for it to operate there. This determination will undoubtedly factor in whether this study will be conducted in a respectful way, honoring the knowledge caregivers provide, as well as incentive structures.
Increasing Response Rates
Knowing that they will receive an incentive for completing a survey or interview increases the likelihood that caregivers will complete data collection activities. Previous studies demonstrated that AIAN people living in urban areas were much less likely to participate in a study if it was led by the federal government and less likely to participate without incentives (Buchwald et al., 2006). Given the mistrust of federally sponsored research in tribal communities, an incentive may be critical in obtaining an adequate response rate. A survey of more than 1000 tribal college students presented vignettes of different types of research studies to gauge the likelihood of participation. Noe et al (2007) found that immediate compensation approximately doubled the odds of participation in focus group and intervention studies and lack of compensation reduced the odds of participation by 20% across all types of studies.
Incentives will be instrumental in retaining representative respondents over time. Caregivers will be asked to complete surveys at three time points over the course of a year and keeping participating caregivers engaged longitudinally will be essential. Caregivers willing to complete initial surveys are likely to grow weary of repeated surveys and need additional motivation with repeated measurement. Our analysis plan includes assessment of caregiver characteristics over time, and poor retention will compromise our ability to draw conclusions from those analyses.
To our knowledge, there are no published experimental studies that show the impact of incentives on response rates or response bias in AIAN communities. Experimental studies where incentives are provided to some research participants but not others would likely not be approved by tribes and their research review boards due to expectations of the equitable treatment of community members. Providing some community members with services or compensation, while denying them to others is not an acceptable practice within tribal communities (Kilburn, 2018). While we don’t have evidence of the effect of incentives on longitudinal response rates from experimental studies, we do know that studies asking participants to respond to hypothetical scenarios suggest that AIAN research participants are more likely to participate in data collection when offered an incentive (Buchwald et al., 2006; Noe et al., 2007).
Our team has extensive experience conducting research in AIAN communities; previously, when we offered incentives, we were successful in obtaining and keeping a diverse sample of participants over time. The MUSE Principal Investigator and Co-Investigator were involved in the American Indian and Alaska Native Head Start Family and Child Experiences Survey 2015 (AI/AN FACES 2015; OMB# 0970-0151) which was funded by ACF and carried out with 21 Head Start programs run by federally-recognized American Indian tribes in the Office of Head Start Region XI. This study utilized respondent incentives and obtained high response rates among parents in this national study. Parents were provided $25 to complete a 30-minute parent survey in the fall of 2015 and spring of 2016; 83% of eligible parents in the fall completed a survey, and 82% of eligible parents in the spring completed a survey. In other recent work, the MUSE Principal Investigator and Co-Investigator conducted studies that offered incentives to AIAN respondents, and those studies were successful at recruiting and retaining study participants. In an NIH-funded study evaluating a substance use prevention program (Thiwáhe Gluwášakapi; Strengthening Families) for young adolescents on a Northern Plains Reservation (R01DA035111; Whitesell, PI), youth and their parents were provided $25 for a survey that took 45-60 minutes to complete. In that study, 88% of parents and 89% of youth were retained to complete a 6-month follow-up survey. In a study with families enrolled in Early Head Start in a tribal community in Oklahoma funded by an ACF grant (Buffering Toxic Stress, 90YR0058; Sarche, PI), parents were provided $25 to complete surveys at study enrollment, and again 3-months and 6-months later. Each survey took approximately 45 minutes; 77% of participants were retained at the 3-month follow-up and 74% at 6-months. These data suggest that our plan to offer incentives to caregivers for completing the three longitudinal caregiver surveys can play a role in successfully recruiting and retaining caregivers for the MUSE study. These experiences inform our selection of the survey incentives. For the one-hour interview, on the other hand, we draw on the literature associated with cognitive interviews, where the respondent is being asked about impressions rather than providing factual information. OMB has allowed $40 incentives for one-hour cognitive interviews.
All study participants will undergo a combined informed consent and HIPAA Authorization process. CAIANH is a HIPAA covered entity and is required by law to comply with HIPAA regulations. All participants must provide their consent prior to enrolling in the study by electronically signing the combined consent and HIPAA Authorization form (Attachment A). Participants will be informed that their information will be kept private to the extent permitted by law. MUSE Team members and grantee staff engaged in data collection will be fully trained in human subjects’ protection and will sign an agreement committing to keep all participant information private (Attachment C). All interviewers and data collectors will be knowledgeable about privacy procedures and will be prepared to describe them in detail or to answer any related questions respondents raise. Staff participants will be assured that their individual survey responses will never be shared with other grantee staff, Federal funders, or anyone else outside the research team.
This study was reviewed and approved by the Colorado Multiple Institutional Review Board (COMIRB). See Attachment D for documentation of initial IRB approval. COMIRB operates under Federalwide Assurance Number 00005070, and thereby adheres to the requirements in the HHS Protection of Human Subjects regulations at 45 CFR Part 46. COMIRB approval for this study is contingent upon receiving approval from OMB and the applicable tribal review entities. Final COMIRB approval will be obtained prior to the beginning of any data collection activities when OMB and tribal approvals are in place. Under the study protocol submitted to COMIRB, the following procedures will be undertaken to protect data privacy:
All data are to be stored on a secure database server which is separate from the web-facing server – a best-practice for internet-based security.
All user access requires unique user accounts and passwords.
All user actions are recorded in a secure audit log.
The database server is routinely backed-up. All security patches and application updates are applied immediately upon release by the developer.
All qualitative interview data will be recorded on encrypted digital recorders, uploaded onto a secure data storage platform, transcribed with identifiers stripped, and all audio files subsequently deleted. Paper data will be stored in a locked, secured cabinet at participating grantee offices only until they are securely transmitted to the MUSE Team for electronic entry or until they can be entered locally into the web-facing server. Once data from the paper form has been either securely transmitted to the MUSE Team or entered electronically, the paper data will be shredded.
The MUSE study will not collect any direct identifiers from caregiver participants. All caregiver data will be submitted with a program ID instead of caregiver names. Grantees will also have the option of submitting staff data with an ID instead of staff name. Individual participants will never be identified in any study dissemination materials or activities. We will assure both caregivers and staff that their responses will be reported only as part of aggregate statistics across all participants. Names of participating grantees may be used in dissemination materials, but grantee-level data will never be publicly reported. It will be made clear that all participants may withdraw their consent at any time and/or refuse to participate in any study activity. Information will not be maintained in a paper or electronic system from which they are actually or directly retrieved by an individuals’ personal identifier.
MUSE will collect administrative data from grantees on topics that could be considered sensitive. Grantees can choose to submit data from depression and substance use screeners. These data are currently collected under Tribal Home Visiting Form 2 (OMB control number 0970-0500, expiration date 8/31/2020)5 to satisfy federal performance measurement requirements. Grantees collect these data as specified by their local data collection protocols and report them in the aggregate to ACF. Given the sensitivity of these data, MUSE will allow grantees to opt into providing participant-level data for both depression and substance use screeners.
MUSE will also collect data directly from home visitors and home visiting participants using quantitative surveys and qualitative interviews. These surveys and interviews will include questions that could be considered sensitive because they ask respondents to make critical appraisals of agency staff (including supervisors) and of relationships between home visitors and families served. Similarly, staff surveys ask grantee staff about job satisfaction, future career plans including possible intent to leave, and working environment, which might be sensitive to some. These questions are critical to answering MUSE study Aims 2 and 3. Responses will not be associated with individuals or grantees in dissemination activities. Qualitative interviews also include a limited number of questions about the impact of the COVID-19 pandemic and related responses on home visiting services. MUSE will allow grantees to opt out of these questions. During the informed consent process, survey respondents and interview participants will be informed of their right to not answer any question(s) or stop participation at any time as well as the processes in place to keep their responses private.
Burden Hours6
Exhibit A.3 shows estimated burden of the information collection, which will take place over approximately 27 months. We are requesting a three-year clearance to account for any delays in data collection.
Caregiver Enrollment Form: Form containing basic information about caregivers; filled out by 93 home visitors across all 17 grantees once per caregiver who consents (estimated to be 14 caregivers per home visitor) to be in the study after consent is obtained; average length of 5 minutes.
Caregiver Survey – Baseline: Survey of 565 caregivers across all 17 grantees at enrollment (baseline); average length of 15 minutes.
Caregiver Survey – 6 & 12 Month Follow-up: Survey of 380 caregivers across all 17 grantees 6 months and 12 months after they take the Baseline Survey; average length of 30 minutes per survey.
Rapid Reflect Self-Completed Home Visit Questionnaire for Caregivers: Self-completed questionnaire completed by 1,136 caregivers across all 17 grantees after an average of 12 selected home visits; average length of 5 minutes.
Rapid Reflect Self Completed Home Visit Questionnaire for Home Visitors: Self-completed questionnaire completed by 93 home visitors after an average of 60 selected home visits; average length of 12 minutes.
Staff Surveys
Home Visitor Survey: Survey of 817 home visitors across all 17 grantees done one time only; average length of 70 minutes.
Program Coordinator/ Manager Survey: Survey of 21 program coordinators/managers across all 17 grantees done one time only; average length of 60 minutes.
Program Director Survey: Survey of 21 program directors across all 17 grantees done one time only; average length of 45 minutes.
Local Program Evaluator Survey: Survey of 30 local program evaluators across all 17 grantees done one time only; average length of 30 minutes.
NOTE: If someone has more than one role, we have a process for only one survey to be administered. See SSB for detail on process.
Program Implementation Survey: Survey of 34 staff in management roles completed in teams (average of 2 people per team) for all 17 grantees done one time only; average length of 15 minutes.
Qualitative Interviews
Qualitative Interviews of Home Visitors: Interviews of 42 home visitors, up to 3 per grantee; average length of 120 minutes.
Qualitative Interviews of Program Coordinators/ Managers and Program Directors: Interviews of 1 program coordinator/manager and 1 program director at each grantee; average length of 90 minutes.
Qualitative Interviews of Local Program Evaluators: Group interviews of 30 local program evaluators, up to 3 at some grantees; average length of 90 minutes.
Qualitative Interviews of Caregivers: Interviews of 51 caregivers, 3 per grantee; average length of 60 minutes.
NOTE: If someone has more than one role, we have a process for only one interview to be conducted. See Supporting Statement B for more details.
Implementation Logs: Log of implementation activities completed by program coordinators/managers on staffing changes, training, family group activities, and supervision completed once a month; average length of 40 minutes.
Administrative Program Data: Electronic compilation and submission of Administrative Program Data by local program evaluators every 6 months; average length of 24 hours per submission, including running reports, compiling and reviewing data and submitting the data file.
Instrument |
Total Number of Respondents |
Annual Number of Respondents |
Number of Responses Per Respondent |
Average Burden Hours Per Response |
Annual Burden Hours |
Average Hourly Wage |
Total Annual Cost |
|
Caregiver Enrollment Form |
93 |
31 |
14 |
.08 |
35 |
$19.80 |
$693 |
|
Caregiver Survey - Baseline |
565 |
188 |
1 |
.25 |
47 |
$10 |
$470 |
|
Caregiver Survey – 6 & 12 Month Follow-up |
380 |
127 |
2 |
.50 |
127 |
$10 |
$1,270 |
|
Rapid Reflect Self-Completed Home Visit Questionnaire for Caregivers |
1,136 |
5681 |
6 |
.08 |
273 |
$10 |
$2,730 |
|
Rapid Reflect Self Completed Home Visit Questionnaire for Home Visitors |
93 |
471 |
66 |
.2 |
620 |
$19.80 |
$12,276 |
|
Home Visitor Survey |
81 |
27 |
1 |
1.17 |
32 |
$19.80 |
$633.60 |
|
Program Coordinator/ Manager Survey |
21 |
7 |
1 |
1 |
7 |
$34.07 |
$238.49 |
|
Program Director Survey |
21 |
7 |
1 |
.75 |
5 |
$34.07 |
$170.35 |
|
Local Program Evaluator Survey |
30 |
10 |
1 |
.5 |
5 |
$39.13 |
$195.65 |
|
Program Implementation Survey |
34 |
11 |
1 |
.25 |
3 |
$34.07 |
$102.21 |
|
Qualitative Interviews of Home Visitors |
42 |
14 |
1 |
2 |
28 |
$19.80 |
$554.40 |
|
Qualitative Interviews of Program Coordinators/ Managers and Program Directors |
34 |
11 |
1 |
1.5 |
17 |
$34.07 |
$579.19 |
|
Qualitative Interviews of Local Program Evaluators |
30 |
10 |
1 |
1.5 |
15 |
$39.13 |
$586.95 |
|
Qualitative Interviews of Caregivers |
51 |
17 |
1 |
1 |
17 |
$10 |
$170 |
|
Implementation Logs |
17 |
91 |
24 |
.67 |
145 |
$34.07 |
$4,940.15 |
|
Administrative Program Data |
17 |
91 |
4 |
24 |
864 |
$39.13 |
$33,808.32 |
|
Total: |
2,240 |
$406.94 |
$59,418.31 |
|||||
1 The annual number of respondents is annualized over 2 years for instruments that are completed by respondents on an ongoing basis. |
|
|||||||
Total Annual Cost
The estimated total annualized cost burden to respondents is based on the burden hours and estimated hourly wage rates for each data collection instrument, as shown in the two right-most columns of Exhibit A-1. These estimates are based on:
an assumed hourly wage of $34.07 for program directors and coordinator/managers, based on mean hourly wage for “Social and Community Service Managers”, as reported in the May 2016 U.S. Department of Labor, Bureau of Labor Statistics, Occupational Employment and Wage Estimates, https://www.bls.gov/oes/current/oes_nat.htm.
an assumed hourly wage of $19.80 for home visitors, based on mean hourly wage for “Community Health Workers,” as reported in the May 2016 U.S. Department of Labor, Bureau of Labor Statistics, Occupational Employment and Wage Estimates, https://www.bls.gov/oes/current/oes_nat.htm.
an assumed hourly wage of $39.13 for local program evaluators, based on mean hourly wage for “Social Scientist or Related”, as reported in the May 2016 U.S. Department of Labor, Bureau of Labor Statistics, Occupational Employment and Wage Estimates, https://www.bls.gov/oes/current/oes_nat.htm.
an assumed hourly rate of $10.00 for caregivers. This equates to annual earnings of $20,800 for a worker employed full-time year-round. For reference, this assumed wage rate is more than the federal minimum wage of $7.25 per hour.
The MUSE Team will provide an honorarium to grantees participating in MUSE in recognition of the administrative burden associated with their participation in the study. Honoraria will range between $1,560 and $3,110 per site, depending on the size of the grantee staff team and number of caregivers enrolled in home visiting, will be split into four equal gifts to be distributed throughout the data collection period with the grantee’s continued participation. The honoraria is less than the anticipated cost of the staff time needed to facilitate MUSE data collection with caregivers.
The total cost for the data collection activities under this current request will be $1,964,349.60. Annual costs to the Federal government will be $654,783 for this proposed data collection.
This is a new data collection.
Analysis Plan
In line with our mixed-method approach, the analytic plan will involve an iterative process of analyzing both qualitative and quantitative data.
Time Schedule and Publication
Data collection will begin once OMB approval is obtained. The evaluation contractor will present findings from analysis of the information collected in a final report, expected in late 2022. OPRE and James Bell Associates, Inc. will publicly disseminate this report. The evaluation contractor will also produce interim reports for participating grantees. Dissemination efforts for MUSE will include professional peer-reviewed journal publications, federal reports, reports back to participating grantees, and professional conference presentations. Each dissemination product will be developed to highlight the findings and recommendations most relevant for the target audience(s) of that product (grantee, federal, scientific, technical assistance, and/or other key stakeholders). A dissemination committee, made up of representatives from participating grantees, will provide input on dissemination priorities; collaborate on analysis, interpretation and dissemination of findings; and approve all final dissemination materials. Exhibit A.4 provides an overview of the study timeline, and Exhibit A.5 outlines a more detailed data collection schedule.
Expected Time Period |
Activity |
During OMB review period |
Preparation for data collection |
0 – 27 months following OMB approval (Approximately Fall 2018-Fall 2020) |
Data Collection
|
12-24 months following OMB approval (Approximately Fall 2019-Fall 2020) |
Interim Data Analyses Interim Reports
|
24-36 months following OMB approval (Approximately Fall 2020-Fall 2021) |
Final Data Analyses
|
36 months following OMB approval (Approximately Fall 2021) |
Final Report and Research Briefs |
|
Time following OMB Approval |
||||||||||
Instrument |
Months 0-6 |
Months 7-12 |
Months 13-18 |
Months 19-24 |
Months 25-30 |
||||||
Caregiver Enrollment Form: Ongoing throughout the first 18 months of the data collection period |
|
|
|
|
|
||||||
Caregiver Survey – Baseline: Ongoing throughout the first 18 months of the data collection period |
|
|
|
|
|
||||||
Caregiver Survey – 6 & 12 Month Follow-up: Ongoing throughout months 7-26 of the data collection period |
|
|
|
|
|
|
|||||
Rapid Reflect Self-Completed Home Visit Questionnaire for Caregivers: Ongoing throughout months 2-24 of the data collection period |
|
|
|
|
|
||||||
Rapid Reflect Self Completed Home Visit Questionnaire for Home Visitors: Ongoing throughout months 2-24 of the data collection period |
|
|
|
|
|
||||||
Home Visitor Survey: Within the first month of the data collection period for all current staff, and as needed for staff hired during the data collection period |
|
|
|
|
|
||||||
Program Coordinator/ Manager Survey: Within the first month of the data collection period for all current staff, and as needed for staff hired during the data collection period |
|
|
|
|
|
||||||
Program Director Survey: Within the first month of the data collection period for all current staff, and as needed for staff hired during the data collection period |
|
|
|
|
|
||||||
Local Program Evaluator Survey: Within the first month of the data collection period for all current staff, and as needed for staff hired during the data collection period |
|
|
|
|
|
||||||
Program Implementation Survey for Managers: Within the first month of the data collection period |
|
|
|
|
|
||||||
Qualitative Interviews of Home Visitors: Ongoing throughout the data collection period |
|
|
|
|
|
||||||
Qualitative Interviews of Program Coordinators/ Managers and Program Directors: Ongoing throughout the data collection period |
|
|
|
|
|
||||||
Qualitative Interviews of Local Program Evaluators: Ongoing throughout the data collection period |
|
|
|
|
|
||||||
Qualitative Interviews of Caregivers: Ongoing throughout the data collection period |
|
|
|
|
|
||||||
Implementation Logs: Ongoing throughout the data collection period |
|
|
|
|
|
||||||
Administrative Program Data: Once every six months throughout the data collection period |
|
|
|
|
|
|
|
|
|
||
Shading Key |
|
||||||||||
|
Ongoing Data Collection |
||||||||||
|
|||||||||||
All instruments will display the expiration date for OMB approval.
No exceptions are necessary for this information collection.
Administration for Children and Families. (2012). Tribal Maternal, Infant, and Early Childhood Home Visiting (TMIECHV) Grant Program under the Affordable Care Act (ACA). Funding Opportunity HHS-2012-ACF-OCC-TH-0302. Washington, DC: U.S. Department of Health and Human Services. Retrieved from http://www.acf.hhs.gov/grants/open/foa/files/HHS-2012-ACF-OCC-TH-0302_0.pdf.
American Academy of Pediatrics, Council on Child and Adolescent Health. (1998). The role of home-visitation programs in improving health outcomes for children and families. Pediatrics. 1998;10:486–489.
Bilukha O, Hahn RA, Crosby A, et al. (2005). The effectiveness of early childhood home visitation in preventing violence. Am J Prev Med. 2005;28(251):11–39.
Buchwald, D., V. Mendoza-Jenkins, C. Croy, H. McGough, M. Bezdek, and P. Spicer. 2006. Attitudes of urban American Indians and Alaska Natives regarding participation in research. J Gen Intern Med 21:648-51.
Castro, F., Barrera, M., & Steiker, L. (2010). Issues and challenges in the design of culturally adapted evidence-based interventions. Annual Review of Clinical Psychology, 6, 213-239,
doi:10.1146/annurev-clinpsy-033109-132032.
Davis, S. M.; Reid, R. . (1999). Practicing participatory research in American Indian communities. The American Journal of Clinical Nutrition 69:755-759.
Del Grosso, P., Kleinman, R., Mraz Esposito, A., Sama-Miller, E., & Paulsell, D. (2012). Assessing the evidence of effectiveness of home visiting program models implemented in tribal communities. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Doughty, M. (2017). Compensation for Study of Participation in Tribal Communities: A Research Note. Indigenous Policy Journal, Vol 28 no (1), 2017.
Duggan, A., Portilla, X., Filene, F., Crowne, S., Hill, C., Lee, H., & Knox, V. (2018). Implementation of Evidence-Based Early Childhood Home Visiting: Results from the Maternal and Infant Home Visiting Program Evaluation. OPRE Report 2018-76A. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Dunn LB, Gordon NE. (2005). Improving Informed Consent and Enhancing Recruitment for Research by Understanding Economic Behavior. JAMA. 2005;293(5):609–612.
Dusenbury, L., Brannigan, R., Hansen, W., Walsh, J., & Falco, M. (2005). Quality of implementation: Developing measures crucial to understanding the diffusion of preventive interventions. Health Education Research, 20 (3), 308-313.
Emanuel, E. J. (2004), Ending Concerns About Undue Inducement. The Journal of Law, Medicine & Ethics, 32: 100-105.
Gomby, D., Culross, P., & Behrman, R. (1999). Home Visiting: Recent Program Evaluations: Analysis and Recommendations. The Future of Children, 9(1), 4-26.
Gottfredson, D., Cook, T., Gardner, F., Gorman-Smith, D., Howe, G., Sandler, I., & Zafft, K.
(2015). Standards of Evidence for Efficacy, Effectiveness, and Scale-Up Research in Prevention
Science: Next Generation. Prevention Science 2015 Oct;16(7):893-926. doi: 10.1007/s11121-015-0555-x
Guyll, Spoth, and Redmond, (2003). The Effects of Incentives and Research Requirements on Participation Rates for a Community-Based Preventive Intervention Research Study. The Journal of Primary Prevention (2003) 24: 25.
Kemp, L. (2016). Adaptation and fidelity: A recipe analogy for achieving both in population scale implementation. Prevention Science, 17 (4), 429-438.
Kilburn, M. R., Lyon, K., Anderson, C., Gutman, P. and Whitesell, N. R. (2018). Methodological Considerations for Home Visiting Research in Tribal Communities. Infant Mental Health Journal, 39: 303-311.
Lo, B. (2010). Ethical Issues in Clinical Research: A practical guide. Philadelphia, PA: Lippincott Williams & Wilkins.
Lyon, K., Geary, E., Sparr, M., Buckless, B., Salvador, M., & Morales, J. (2015). Tribal Maternal, Infant, and Early Childhood Home Visiting: A report to Congress (OPRE Report #2015-88).
Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Mihesuah, D. A. (1993). Suggested guidelines for institutions with scholars who conduct research on American Indians. American Indian Culture and Research Journal 17:131-139.
Nielsen, M. O. and L. A. Gould. (2007). Non-Native scholars doing research in Native American communities: A matter of respect. The Social Science Journal 44:420-433.
Noe, T. D., S. M. Manson, C. Croy, H. McGough, J. A. Henderson, and D. S. Buchwald. (2007). The influence of community-based participatory research principles on the likelihood of participation in health research in American Indian communities. Ethn Dis 17:S6-14.
Pacheco, C. M., Daley, S. M., Brown, T., Filippi, M., Greiner, K. A., & Daley, C. M. (2013). Moving Forward: Breaking the Cycle of Mistrust Between American Indians and Researchers. American Journal of Public Health, 103(12), 2152–2159.
Roberts, E., Morales, J., Salvador, M., Lyon, K., Geary, E., & Buckless, B. (2018). Grantee-led evaluations in the Tribal Maternal, Infant, and Early Childhood Home Visiting Program: A compilation of grantee evaluation plan profiles, OPRE Report # XXX, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Sahota, P.C. (2010). Community Based Participatory Research in American Indian and Alaska Native Communities. National Congress of American Indians Policy Research Center.
Sobeck, J. L., E. E. Chapleski, and C. Fisher. (2003). Conducting Research with American Indians. Journal of Ethnic & Cultural Diversity in Social Work 12:69-84.
Spillane, J., Reiser, B., & Reimber, T. (2002). Policy implementation and cognition: Reframing and refocusing implementation research. Review of Educational Research, 72 (3), 387-431.
Stoltzfus, E., & Lynch, K. (2009). Home visitation for families with young children. Washington, DC: Congressional Research Service.
Tribal Evaluation Workgroup. (2013, September). A roadmap for collaborative and effective evaluation in tribal communities. Washington, DC: Children’s Bureau, Administration for Children and Families, U.S. Department of Health and Human Services.
U.S. Department of Health and Human Services, Administration for Children and Families, Office of Child Care. (no date). Guidance for submitting a needs assessment and plan for responding to identified needs (Phase 2 implementation plan) Cohort 3 grantees. Retrieved from http://www.acf.hhs.gov/sites/default/files/occ/tribal_hv_implementation_plan_guidance_cohort_3_revised_final_508.pdf.
U.S. Department of Labor, Bureau of Labor Statistics (May 2016). Occupational Employment and Wage Estimates, https://www.bls.gov/oes/current/oes_nat.htm.
Wandersman, A., Imm, P., Chinman, M., & Kaftarian, S. (2000). Getting to outcomes: A results-based approach to accountability. Evaluation and Program Planning, 23(3), 389-395.
1 https://www.acf.hhs.gov/opre/resource/implementation-evidence-based-early-childhood-home-visiting-results-mother-infant-home-visiting-program-evaluation
2 The Administration for Children & Families Office of Planning, Research, and Evaluation (March 2016). The Administration for Children & Families Common Framework for Research and Evaluation. Retrieved from https://www.acf.hhs.gov/sites/default/files/opre/acf_common_framework_for_research_and_evaluation_v02_a.pdf
3 The Tribal MIECHV program will be submitting a renewal OMB package for demographic and service utilization data in early 2019.
4 The Tribal MIECHV program will be submitting a renewal OMB package for the performance measurement data in late 2019.
5 The Tribal MIECHV program will be submitting a renewal OMB package for the performance measurement data in late 2019.
6 When calculating burden, we estimated a higher number of respondents per instrument than for our sample estimates. The additional burden is requested in case grantees enroll more caregivers, complete more home visits, or retain caregivers at a higher rate than we estimated based on available data.
7 We estimate that fewer home visitors will complete the Home Visitor Survey (n=81) than the Caregiver Enrollment Form (n=93) because home visitors will only receive a survey after they have been in their position for four months. Home visitors will begin completing Caregiver Enrollment Forms as soon as they begin serving families.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |