Supporting Statement: Part A
Formative Research on Community-Level Factors that Promote the Primary Prevention of Adverse Childhood Experiences (ACEs) and Opioid Misuse Among Children, Youth, and Families in Tribal American Indian and Alaska Native (AI/AN) Communities
OMB# 0920-20PJ
April 16, 2021
Point of Contact
Jessica Elm, MSW PhD
Contact Information
Centers for Disease Control and Prevention
National Center for Injury Prevention and Control
4770 Buford Highway NE
Atlanta, GA 30341-3724
Phone 770-488-6727
Email qsj3@cdc.gov
CONTENTS
Section Page
Summary table 3
Justification 3
A.1. Circumstances Making the Collection of Information Necessary 3
A.2. Purpose and Use of Information Collection 5
A.3. Use of Improved Information Technology and Burden Reduction 8
A.4. Efforts to Identify Duplication and Use of Similar Information 9
A.5. Impact on Small Businesses or Other Small Entities 9
A.6. Consequences of Collecting the Information Less Frequently 9
A.7. Special Circumstances Relating to the Guidelines of
5 CFR 1320.5(d)2 10
A.8. Comments in Response to the Federal Register Notice and
Efforts to Consult Outside the Agency 10
A.9. Explanation of Any Payment or Gift to Respondents 11
A.10. Protection of the Privacy and Confidentiality of Information
Provided by Respondents 12
A.11. Institutional Review Board and Justification for Sensitive Q …. 13
A.12. Estimates of Annualized Burden Hours and Costs 14
A.13. Estimates of Other Total Annual Cost Burden to Respondents
or Record Keepers 16
A.14. Annualized Cost to the Government 16
A.15. Explanation for Program Changes or Adjustments 17
A.16. Plans for Tabulation and Publication and Project Time Schedule 17
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate 18
A.18. Exceptions to Certification for Paperwork Reducation Act
Submissions 18
References 18
ATTACHMENTS
A Authorizing Legislation: Public Health Service Act
B Federal Register Notice
B1 Public Comments
C Information Letter
D Telephone Screening
E Confirmation Letter/Email
F Informed Consent
G Reminder Email
H List of Local and National Resources for Participants
I Demographic Survey
J Interview - Focus Group for Tribal Elders-Tribal Leaders-Traditional Healers
K Focus Group Discussion Guide for Parents-Caregivers and Young Adults
L Interview_Discussion Guide for Community Leaders and Service Providers
M Privacy Impact Assessment (PIA)
N IRB Letter
O Study Flyer
Goal of the study: The goal of this Information Collection
Request (ICR) is to conduct formative qualitative studies to
identify community-level protective factors (e.g., intergenerational
connection to Tribal Elders) and primary prevention strategies
across a range of Tribal communities (i.e., urban and
rural/reservation) to prevent adverse childhood experiences (ACEs)
and opioid misuse.
Intended use of the resulting
data: To identify ways to prevent ACEs and opioid misuse before
they occur, provide information regarding best available evidence
at the outer levels of the social ecology, and extend the National
Center for Injury Prevention and Control’s current
investments on identifying community-level resources, supports, and
prevention strategies for children and their families in Tribal
communities.
Methods
to be used to collect: A qualitative study will be used to
collect data using in-person (or virtual) semi-structured focus
groups/interviews. A brief demographic survey will be included.
Sampling will be purposive and include key respondent groups (i.e.,
young adults, parents and caregivers of American Indian/Alaska
Native (AI/AN) children, Tribal Elders, and community leaders and
service providers) in Tribal urban and rural/reservation
communities. The subpopulation to be
studied: Local people (e.g., parents and caregivers of AI/AN
children, Tribal Elders) living in Tribal urban and
rural/reservation communities, including adults 18 years or older
affected by the opioid epidemic.
How data will
be analyzed: Data will be analyzed using well-established
qualitative analysis methods, such as coding interviews for themes
about perceptions of community level protective factors (e.g.,
access to primary prevention services).
SUMMARY TABLE
A. JUSTIFICATION
A.1. Circumstances Making the Collection of Information Necessary
The Centers for Disease Control and Prevention (CDC), National Center for Injury Prevention and Control (NCIPC), Division of Violence Prevention (DVP) requests the Office of Management and Budget (OMB) approval for three years for a new information collection request (ICR) to collect and analyze community-level protective factors and primary prevention strategies (e.g., collective healing approaches) that might be effective at preventing adverse childhood experiences (ACEs) and opioid misuse while also providing safe, stable, and nurturing relationships and environments for all American Indian/Alaska Native (AI/AN) children and their families. Qualitative studies conducted under this ICR are consistent with the research priorities of CDC NCIPC (CDC, 2019) and DVP Strategic Priority to prevent multiple forms of violence (CDC, 2016). The data collections supported under this ICR will be used to understand community-level protective factors and primary prevention strategies for ACEs and opioid misuse in urban and rural Tribal communities.
ACEs are preventable, potentially traumatic events that occur in childhood (0-17 years) such as experiencing violence, abuse, or neglect; witnessing violence in the home; and having a family member attempt or die by suicide (Felitti et al., 1998; National Child Traumatic Stress Network, 2019). There is a robust evidence base linking ACEs to a variety of poor health outcomes across the life span, including depression, alcohol misuse, and substance use disorder (Derenfinko et al., 2019; Elm, 2020; Stein et al., 2017; Warne et al., 2017), suicide, chronic diseases such as cancer and heart disease (Felitti et al., 1998; Gilbert et al., 2015; Merrick et al., 2019), and violence perpetration and victimization (Charles et al., 2003; Ports et al., 2016; Schofield et al., 2013). The ongoing opioid epidemic is a complex and significant public health crisis that exposes children to opioid misuse, violence, and other ACEs, and challenges the ability of health and human service systems to mitigate the effects of opioid misuse and ACEs on children and families across the U.S. (Birnbaum et al., 2011; Meyer et al., 2014).
AI/AN populations experience a disproportionate burden of opioid misuse and ACEs, and ACE-related health outcomes, including opioid overdose, sexual assault, and suicide attempts (Anda et al., 2008; Breiding et al., 2014; Brockie et al., 2015; Dube et al., 2001; Jones et al., 2020; Kenny & Singh, 2016; Mersky et al., 2018; Ports et al., 2016; Walls et al., 2020; Warne et al., 2017). The nature and consequences of ACEs in Tribal communities is unique because of historical trauma and stark socioeconomic disparities (Grayshield et al., 2015; Hamby et al., 2020). In addition, there are gaps in the provision of adequate healthcare (Whitesell et al., 2018). Both the complex nature of trauma and lack of services contribute to higher rates of substance misuse in Tribal communities (Whitesell et al., 2012). AI/AN populations have historically been understudied (Elm, 2020).
This study addresses critical research gaps and extends efforts to prevent violence and other ACEs before they occur and to build evidence of effectiveness of community-level strategies and approaches at the outer levels of the social ecology to Tribal communities. Even though AI/AN populations are vulnerable to ACEs and opioid misuse, Tribal communities are resilient with many cultural strengths and protective factors that might create the conditions for safe, stable, and nurturing relationships and environments. Potential community-level protective factors such as close networks, intergenerational connection to Tribal Elders, spirituality, and collective healing approaches have endured over time to support health and wellbeing (Antone & Arambula Solomon, 2016; Elm et al., 2016; Henson et al., 2017; Wright, 2013). There is a need to understand and capture a range of community-level protective factors within Tribal communities (Elm et al., 2019). This study will work with Tribal communities and regional liaisons to honor local voices and cultural protocols to learn best practices that are beneficial to Tribal communities in preventing ACEs and opioid misuse.
This study will use qualitative methods to conduct semi-structured focus groups/interviews along with a brief demographic survey with respondents (e.g., young adults and Tribal Elders) affected by the opioid epidemic in Tribal urban and rural/reservation communities. CDC intends to use findings from this study to strengthen violence prevention efforts in impacted communities and geographic areas. We anticipate this study will inform primary prevention efforts within Tribal communities related to the intersections of violence, ACEs, and opioid misuse in order to provide safe, stable, and nurturing relationships and environments for all AI/AN children and their families. The study’s focus on Tribal communities helps fill gaps in prior (e.g., Feder at al., 2018) and current research such as the CDC and ACF formative studies on opioids and their impact on children and families that do not contain adequate information on AI/AN populations. Identifying community-level prevention strategies can work to improve the conditions in which children and families live and may be effective at protecting children from harm while also disrupting potential developmental trajectories toward substance misuse, victimization and perpetration of violence, and poor health (CDC, 2019). We anticipate that this qualitative study will explore barriers and facilitators that hinder or promote ACEs and opioid misuse prevention at a much deeper level than existing quantitative data collections (e.g., Sapp & Hooten, 2019; Romanowicz et al., 2019). Only qualitative data collections can explore the “Whys” and “Hows” of the successes and failures of ACEs and opioid misuse prevention efforts, and how the lives of people are directly impacted by specific programs and prevention activities. This ICR describes methods that will be used to obtain detailed in-depth assessment of prevention efforts among AI/AN people living in urban and rural/reservation Tribal communities in the U.S.
Authority for CDC’s National Center for Injury Prevention and Control (NCIPC) to collect these data is granted by Section 301 of the Public Health Service Act (42 U.S.C. 241). This act gives federal health agencies, such as CDC, broad authority to collect data and participate in other public health activities, including this type of program implementation evaluation (Attachment A).
A.2. Purpose and Use of Information Collection
The purpose of this ICR is to enable CDC NCIPC to conduct formative qualitative studies to identify community-level protective factors (e.g., intergenerational connection to Tribal Elders) and primary prevention strategies across a range of Tribal communities (i.e., urban and rural /reservation) to prevent ACEs and opioid misuse. Populations for this data collection include AI/AN people (i.e., young adults, parents and other caregivers, Tribal Leaders and Elders, community-based service providers, and community leaders) living in urban and rural communities that are affected by the U.S. opioid epidemic.
Results from this data collection will be communicated to relevant public health officials and community stakeholders in the study locations. Relevant public health officials might include local health department personnel involved with violence prevention and opioid misuse prevention, care, and treatment. Relevant community stakeholders might include non-governmental healthcare and social service providers, Tribal entities and Leaders, or staff of community-based organizations serving Tribal communities (see Table 1. Respondents and Data Collection by Tribal Community). These local public health officials and community stakeholders will use the study results to guide strategies to further strengthen their local prevention efforts within their regions.
Data collection methods used in this qualitative study include well-established qualitative methods, including in-depth open-ended individual interviews and focus groups and a brief demographic survey, using pen and paper (in person) or through an online platform. No personal identifiable information (PII) will be collected. All respondents will provide informed consent (Attachment F) and will be told that participation is voluntary. Focus groups/interviews will be conducted after consent is obtained at the designated date, time, and location. In-depth interview and focus group guides will include questions and probes designed to collect information pertinent to the following research questions.
Community or Societal Level Questions
What supports, resources, services, and other factors are available in Tribal communities located in urban and rural/reservation areas that may be protective against children’s exposure to violence and other ACEs and potentially lessening impacts on opioid misuse?
How are these community and/or societal level services similar or different in urban and rural/reservation areas?
What are factors that may limit the availability and accessibility of community or societal level services in urban and rural/reservation areas?
What primary prevention strategies exist in Tribal communities located in urban and rural/reservation areas that could be feasibly implemented in similar urban and rural/reservation AI/AN settings to prevent children’s exposure to violence and other ACEs and potentially lessening impacts on opioid misuse?
Family- and Individual-Level Questions
What are protective factors for AI/AN children in urban and rural/reservation areas that prevent exposure to violence and other ACEs and opioid misuse?
How are these protective factors similar or different in urban and rural/reservation areas?
What are factors that may challenge the availability and accessibility of protective factors for AI/AN children, youth, and families in urban and rural/reservation areas?
What are key components of a person and/or family’s healing journey to overcome trauma such as children’s exposure to violence and other ACEs and opioid misuse?
How are these key components similar or different in urban and rural/reservation areas?
What are factors that may challenge a person and/or a family’s healing journey?
This qualitative research study will provide CDC NCIPC with:
Greater understanding of community resources or supports for prevention of ACEs and opioid misuse in AI/AN populations
Greater understanding of prevention strategies that may be transferrable to other similar communities (i.e., similar urban or rural/reservation Tribal communities)
Data to understand trajectories of individuals who have completed a healing journey and individuals who are currently working through or who have not yet started a healing journey (e.g., trajectory of violence and/or opioid misuse)
The research team will conduct site visits to urban and rural Tribal communities in two diverse geographic regions for recruitment of respondents and data collection. There will be a total of four Tribal communities that participate in the study: one urban and one rural/reservation community in the Pacific Northwest region (e.g., Washington state) and one urban and one rural/reservation community in the Great Lakes region (e.g., Minnesota). Academic partners familiar with the tribal communities in these areas will act as liaisons to confirm the level of confidentiality that each Tribal community requests. These two regions have higher opioid overdose mortality rates relatively to other areas in Indian Country (Tipps, Buzzard, & McDougall, 2018). The aim is to conduct all data collection in person during site visits to each of the four communities. However, it is anticipated that scheduling conflicts may necessitate conducting some interviews over the phone after the site visit to complete all data collection. Due to COVID-19, at the time of the focus groups/interviews, social distancing and public health safety measures will be implemented, including considerations for phone/virtual meetings instead of in-person sessions.
Protections of Tribes and Confidentiality
On January 26, 2021 the White House issued a Memorandum for the Heads of Executive Departments and Agencies titled “Memorandum on Tribal Consultation and Strengthening Nation-to-Nation Relationships”. This memorandum acknowledged and recommitted federal agencies to respect “tribal sovereignty and self-governance, commitment to fulfilling Federal trust and treaty responsibilities to Tribal nations…”. Moreover, the memo states that, “we best serve Native American people when Tribal governments are empowered to lead their communities, and when Federal officials speak with and listen to Tribal leaders in formulating Federal policy that affects Tribal Nations.” (Biden, 2021). This memo is supported by Executive Order No. 13175.
CDC’s Tribal Consultation Policy was last updated in 2013 and states that “Indian Tribes have an inalienable and inherent right to self-government. As sovereign nations, Indian Tribes exercise inherent sovereign powers over their members [emphasis added], territory, and lands. As a Federal organization, CDC recognizes its special commitment and unique relationship with Indian Tribes and is committed to fulfilling their critical role in promoting the health and safety of Indian Tribes.” (CDC, 2013).
The sovereign status of Federally Recognized Indian Tribes (FRTs) is further recognized in Title 45 Public Welfare Department of Health and Human Services, Protection of Human Subjects - also known as the Common Rule. For example, Subpart A, §46.101 states that Tribal laws must be followed and such laws may include additional protections in human subjects research (45 CFR 46.101(f), Subpart A).
There are currently 574 FRTs. The listed Indian entities are acknowledged to have the immunities and privileges available to federally recognized Indian Tribes by virtue of their government-to-government relationship with the United States as well as the responsibilities, powers, limitations, and obligations of such Tribes. A full list of FRTs can be found at: https://www.federalregister.gov/documents/2021/01/29/2021-01606/indian-entities-recognized-by-and-eligible-to-receive-services-from-the-united-states-bureau-of
A recruitment/informational flyer (Attachment O) will be posted at local community organizations (e.g. health clinic, community center) and provide study and contact information for potential participants. Telephone screening of interested individuals will be used to determine eligibility for participation (e.g., affected by the opioid epidemic - Attachment D). Respondents will provide written informed consent (Attachment F), complete a demographic survey (Attachment I), and participate in a focus group or individual interview (in-person or virtually if necessary - Attachments J-L), depending on what is most culturally appropriate and feasible for recruitment, scheduling, and data collection for the community, and also in accordance to COVID-19 guidance. Focus groups/interviews will be conducted with adults aged 18 years or older affected by the opioid epidemic. Table 1. Respondents and Data Collection by Tribal Community provides an overview of respondents and data collection by urban and rural/reservation Tribal community. Respondents will complete a survey on demographic information prior to their focus group/interview.
Table 1. Respondents and Data Collection by Tribal Community
Area |
|
Parents/ Caregivers |
Community Leaders & Service Providers |
Elders/ Tribal Leaders/ Traditional Healers |
Young Adults |
1 |
Rural/ Reservation |
1 focus group (8 people) |
3 in-depth qualitative interviews |
1 focus group (8 people) and 3 in-depth interviews |
1 focus group (8 people) |
1 |
Urban |
1 focus groups (8 people) |
3 qualitative interviews |
1 focus group (8 people) and 3 in-depth interviews |
1 focus groups (8 people) |
2 |
Rural/ Reservation |
1 focus groups (8 people) |
3 qualitative interviews |
1 focus group (8 people) and 3 in-depth interviews |
1 focus groups (8 people) |
2 |
Urban |
1 focus groups (8 people) |
3 qualitative interviews |
1 focus group (8 people) and 3 in-depth interviews |
1 focus groups (8 people) |
Note: 12 focus groups and 24 in-depth interviews; 120 participants
The focus groups/interviews will use discussion guides that will be adapted from the Formative Research on Opioids and Their Impact on Children, Youth, and Families study (OMB#0990-0421). The survey is comprised of demographic items adapted for AI/AN populations.
All protocols for studies conducted under this ICR have undergone Human Subjects Protection regulation review at WCG IRB. Protocols may undergo further Human Subjects Protection - and community- protections review according to procedures of each Tribal entity.
A.3. Use of Improved Information Technology and Burden Reduction
This ICR involves the use of qualitative methods to quickly collect timely data to understand issues affecting AI/AN populations with the greatest risk for ACEs and opioid misuse in specific communities within the U.S. Focus groups/interviews will be supported by digital audio recordings using handheld devices with the consent of the respondents. If any respondent does not agree to being audio recorded, research staff will take notes during the focus group/interview. Paper copies will be used for informed consent and survey completion (in person or through a secure online platform or by phone if necessary). The focus groups/interviews and written surveys will collect only the minimum information necessary for the purposes of the research. Research staff will prioritize questions in the discussion guides that are most central to the study and to the respondent’s experiences and thus cannot be addressed through other sources.
Focus groups/interviews will be scheduled to occur at a time that is convenient for the respondents and requests to reschedule will be accommodated. It is possible that respondents may not be available during the scheduled site visit and/or that some sessions may need to be completed after the site visit. In these cases, the research team will conduct interviews remotely by phone or via a secure online platform to maximize the potential for participation and reduce burden.
A.4. Efforts to Identify Duplication and Use of Similar Information
The ICR represents a new effort to identify protective factors and strategies to prevent ACEs and substance misuse (e.g., opioid use disorder) in urban and rural/reservation Tribal communities. The generic collection that CDC NCIPC is conducting with the Assistant Secretary for Planning and Evaluation (ASPE) entitled, “Addressing substance use disorders among families involved with the child welfare system: A cross-agency collaboration” (OMB# 0900-0421) has a shared goal of identifying promising strategies and potential factors that may protect children and families from ACE exposure and opioid misuse and data collection in high burden rural/reservation communities. The generic collection, however, does not contain adequate information on Tribal populations. The current data collection request addresses this critical information gap. We are in regular communication with ASPE and will share findings from this collection with ASPE.
CDC NCIPC reviewed the Federal Register through RegInfo.gov for duplicate data collection efforts and reached out to other federal agencies that have an interest in Tribal communities, including the National Institutes of Health (NIH) and the Administration for Children and Family (ACF). Two studies from ACF that engage in relevant but not redundant efforts to the current data collection request are (1) “Multi-Site Implementation Evaluation of Tribal Home Visiting” (OMB# 0970-0521), a collection that examines how home visiting programs are operating across diverse Tribal community contexts and identifies factors that lead to successful program implementation; and (2) “OPRE Evaluations: Head Start Family and Child Experiences Survey (FACES 2019) (OMB# 0970-0151), a collection of data on children’s development and home/classroom contexts for AI/AN children and families receiving services from Head Start. We are in regular communication with ACF and will share findings from this collection with ACF as well as with NIH.
A.5. Impact on Small Businesses or Other Small Entities
Some data collection involves respondents (i.e., service providers) from private agencies, such as health care practices; however, these data collections will not have a significant impact on the agencies or entities due to the relatively small number of service providers involved.
A.6. Consequences of Collecting the Information Less Frequently
The data collection request is for a one-time data collection. No respondent will be asked to participate in more than one focus group/interview or survey, and there will be no recurrent or additional data collections under this OMB request. The data collection request involving Tribal communities is a direct response to CDC NCIPC’s efforts to prevent ACEs and substance misuse before they occur and to build evidence of effectiveness at the outer levels of the social ecology in order to provide safe, stable, and nurturing relationships and environments for all children including AI/AN children and their families.
The data collection request involves individuals affected by the opioid epidemic in multiple rural/reservation and urban Tribal communities. If data are not collected, we will be missing key opportunities to learn about promising strategies to support families, prevent violence, ACEs, and opioid misuse; and leverage federal, state, or local opportunities to address challenges across a range of Tribal communities. Without the data collection, we will be missing the voices and perspectives of AI/AN stakeholders (e.g., parents/caregivers, young adults) in developing prevention strategies, which limits our ability to understand the nuances of implementing promising strategies in Tribal communities. Focus groups/interviews with individuals who have firsthand knowledge of and experience with promising strategies will provide insights into the contextual and implementation factors that may not be described in sufficient detail in the literature or in ways that would inform adaptation for Tribal communities.
A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are no special circumstances with this information collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.
A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
A.8.a) Federal Register Notice
A 60-day Federal Register Notice was published in the Federal Register on July 2, 2020 Volume 85, Number 128, pp 39910-39911(Attachment B). CDC received two anonymous and one non-substantive public comment (Attachment B1). No changes were made to the collection.
A.8.b) Efforts to Consult Outside the Agency
CDC NCIPC consulted with the following CDC and non-CDC agencies and entities (see Table 2. Consultation) for the development of the data collection request and recommendations were incorporated into the study design and methodology.
Table 2. Consultation
Agency |
Entity |
CDC |
NCIPC: Molly Evans, MD, MPH, Medical Officer Division of Overdose Prevention: Andrew Terranella, MD, MPH, Sarah Bacon, PhD, Behavioral Scientist Office of the Director, Center for State, Tribal, Local, and Territorial Support (CSTLTS): Delight Satter, MPH, Senior Health Scientist, Tribal Research Program (Confederated Tribes of Grand Ronde); Jessica Damon, MLS, MPH, Senior Program Management Officer National Center for Chronic Disease and Health Promotion, Division of Diabetes Prevention: Dawn Satterfield, PhD, Health Education Specialist
|
ACF |
Office of Planning, Research and Evaluation: Aleta Meyer, PhD, Senior Social Service Research Analyst and Team Lead for Community-Engaged and American Indian/Alaska Native Research in the Division of Family Strengthening
|
Indian Health Services
|
Division of Behavioral Health: Captain Andrew Hunt, MSW, LICSW (Lumbee) Acting Deputy Director
|
Johns Hopkins Bloomberg School of Public Health
|
The Johns Hopkins Center for American Indian Health: Melissa Walls, PhD, (Boise Forte Ojibwe) Director of Great Lakes Hub and Jessica Elm, PhD, Postdoctoral Researcher (Oneida Nation and Stockbridge-Munsee Band of the Mohicans)
|
NIH
|
Division of Program Coordination, Planning, and Strategic Initiatives: David Wilson, PhD, Director, Tribal Health Research Office (Navajo)
|
University of North Dakota |
School of Medicine and Health Sciences: Donald Warne, MD, MPH, Professor of Family and Community, the Associate Dean of Diversity, Equity and Inclusion as well as the Director of the Indians Into Medicine and Master of Public Health Programs (Oglala Lakota)
|
A.9. Explanation of Any Payment or Gift to Respondents
We will provide all respondents with a token of appreciation of $37.50/hour for participating in a two-hour focus group/interview ($75 total) to offset the costs of participating in the study, including costs of transportation and childcare, that may otherwise preclude participation. The aim is to conduct all data collection in person during site visits to each of the four communities, following COVID-19 social distancing guidance when necessary. Respondents for this data collection are individuals in the U.S. aged 18 years and older affected by the opioid epidemic from urban and rural/reservation Tribal communities in two diverse geographic regions. Eligible adults from six participation groups (parents/caregivers, Tribal Elders, community leaders, service providers, Tribal Leaders, traditional healers, and young adults) will be recruited to participate in a two-hour focus group or interview and complete a brief demographic survey. Respondents will be asked to travel to the focus group/interview location. In the event it is necessary to conduct the focus groups/interviews virtually, respondents will be asked to travel to a facility where internet access can be obtained. Following COVID-19 guidance, at the time of the interview/focus group, social distancing and public health safety measurement will be implemented, including considerations for phone/virtual meetings instead of in person.
A Cochrane Collaboration systematic review of randomized controlled trials found that a monetary token of appreciation doubled the response rates of questionnaire returns (Edwards et al., 2002). Studies indicate that tokens of appreciation are an effective method for increasing the response rates of hard-to-reach populations (e.g., Shaghai et al., 2011), respondents from racial/ethnic backgrounds (e.g., Beebe et al., 2005; Dykema et al., 2012), and respondents with lower education typically underrepresented in surveys (e.g., Singer et al., 2000). These tokens of appreciation are to help avoid non-response bias and ensure that economically disadvantaged respondents are not systematically discouraged from participating due to the costs of participation. The token of appreciation follows OMB guidelines for a focus group that imposes burden on respondents in traveling to the facility and in the level of effort required for the focus groups/interviews (e.g., respondents are asked to explain their mental processes as they hear the question and discuss its meaning).
A.10. Protection of the Privacy and Confidentiality of Information Provided by Respondents
The Office of the Chief Information Officer at the CDC has determined that the Privacy Act does apply. The Privacy Impact Assessment (PIA) is attached (Attachment M). Only research staff involved with the site visits and data collection will know the identities of respondents and obtain limited PII in terms of name and contact information (e.g., personal phone number, email address) for recruitment and scheduling purposes. All contact information will be contained in a secure password-protected datafile accessible only to research staff during the data collection process, stored separately and not linked to the research data, and deleted upon data collection completion. No PII will be shared with or accessible to CDC NCIPC.
For data collection, a unique study generated site identification number will be assigned to each community so that information cannot be associated with any individual or community name to reduce the risk of potential harm from breach of confidentiality. During the focus groups/interviews, only the first names of respondents will be used to establish rapport. The research team will create a digital audio recording of the focus group/interview if respondents consent to the recording. The audio recordings will be securely transmitted to a professional transcription firm and not shared with CDC NCIPC. The contractor will instruct the firm that transcribes the recordings to scrub names and any other PII to prepare de-identified transcripts. Accordingly, only the study respondent’s identification number and research data will be available for data analysis. No respondents will be identified by name or description that may disclose their identity and specific quotes will not be attributed to any respondent in any report or other information from the study that becomes publicly available. All data information will be provided as an aggregate summary.
Participation in the research activities is strictly voluntary. The confidentiality of all participating individuals and Tribal communities will be strictly maintained. Respondents will be given as much time as needed to review, ask questions, and complete a written informed consent form before beginning data collection (Attachment F). The consent form includes the following information: explanation of the purposes of the research and description of research procedures; explanation that their participation is voluntary; that they may decline to answer questions in the focus group/interview or questionnaire and stop at any time without consequences; that there are no risks to participating beyond experienced in daily life: the length of time required for the information collection; explanation of the token of appreciation that will be provided; explanation and permission to create an audio recording; and the name and contact information for individuals if there are questions about the study or their rights as a research respondent. Respondents will also be informed that the data they provide will be analyzed and reported in aggregate form for purposes stated in the consent form and treated in a secure manner to the extent allowed by law. The only exception is if there is a federal, state, or local law that requires disclosure (such as to report child abuse) or if a respondent reports plans to harm him/herself or others. Individuals or Tribal communities will not be identified by name in reports of study findings unless agreed upon and approved by the Tribal community, a goal of the study is to disseminate information about promising strategies, which will likely necessitate naming the Tribal community’s location (e.g., Great Lakes Region).
All CDC NCIPC employees involved in reviewing and approving and contractors involved in research activity contract deliverables will be required to attend annual security and privacy training, to sign a nondisclosure agreement and notice about data use policies if required by tribal partner sites, and to update their security and privacy agreements and training on an annual basis. CDC NCIPC employees will not be directly engaged in research activities.
A.11. Institutional Review Board (IRB) and Justification for Sensitive Questions
IRB Approval
CDC has received IRB approval through WCG’s IRB. WCG’s IRB (IRB IRB00000533) and the award subcontractor conducting the research (James Bell and Associates FWA00005415) meet all of the Federal requirements as specified in 45 CFR 46 and are registered with the Office for Human Research Protections and with Federal Wide Assurance, respectively. A copy of the approval letter is provided in Attachment N. We will address human subjects- and community- protections according to each Tribal entity’s human subjects- and community- protections procedures.
Sensitive Questions
The topics that will be covered by this information collection request involve ACEs, misuse of prescribed and non-prescribed opioid substances, and other matters that are commonly considered sensitive or private. We will obtain a certificate of confidentiality which is automatically granted for CDC-funded research to support the research team in protecting the data against compulsory legal demands (e.g., court orders, subpoenas). A certificate of confidentiality indicates that research staff cannot disclose information or documents pertaining to the data collection to anyone else who is not connected with the research. In addition, information may not be disclosed to any federal, state, or local civil, criminal, administrative, legislative, or other action, suit, or proceeding.
The reasons for collection of sensitive information is CDC NCIPC’s interest in learning about protective factors related to children’s exposure to violence and other ACEs and trajectories into opioid misuse and their application in preventing violence to prevent ACEs, and substance misuse in AI/AN populations. As explained in the informed consent, respondents may decline to respond to any question or end participation at any time. All questions, including sensitive questions, are not required and voluntary in nature. Respondents will also be told that their responses are for analysis purposes only and their data will be kept secure and not linked to them personally. The informed consent contains contact information for both the study and the CDC IRB in case a respondent has further questions or concerns.
Respondent safety remains the primary concern for any data collection containing sensitive questions. The research team will be trained and instructed on crisis procedures should one arise and instructions for when respondents become uncomfortable, distressed, or wish to discontinue. All respondents will be provided with a community-specific list of mental health care referral services that they may consult as needed (Attachment H).
A.12. Estimates of Annualized Burden Hours and Costs
The number of individuals to be contacted for recruitment is 160 with an expected response rate for participation of 75% or greater of eligible individuals. The focus group/interview sessions will include 120 respondents. Data collection is a one-time effort. Respondents will participate in either a focus group or an individual interview - not both. Following COVID-19 guidance, at the time of the interview/focus group, social distancing and public health safety measurement will be implemented, including considerations for phone/virtual meetings instead of in person. The total number of burden hours associated with this data collection is 441 hours (see Table 3. Estimated Annualized Burden Hours for details about how this estimate was calculated). The estimate includes 5 minutes for the information letter, 25 minutes for the telephone screening, 5 minutes for the confirmation email, and 5 minutes for the reminder email, 25 minutes to complete the written consent form, 25 minutes to complete the survey, and 2 hours for the focus group/interview. For each of the four sites, there will be one focus group of 8 parents/caregivers, one focus group of 8 and 3 individual interviews with Tribal Elders/Tribal Leaders/Traditional healers, 3 individual interviews with community leaders or service providers, and one focus group of 8 young adults.
Table 3. Estimated Annualized Burden Hours
Type of Respondents |
Data Collection |
Number of Respondents |
Number of Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden (in hours) |
Adults 18 years or older affected by the opioid epidemic (e.g., parents /caregivers of AI/AN children, Tribal Elders) living in Tribal urban and rural/ reservation communities
|
Information Letter (Attachment C) |
160 |
1 |
5/60 |
14 |
Telephone screening (Attachment D) |
160 |
1 |
25/60 |
67 |
|
Confirmation email/letter (Attachment E) |
120 |
1 |
5/60 |
10 |
|
Reminder email (Attachment G) |
120 |
1 |
5/60 |
10 |
|
Informed Consent (Attachment F)
|
120 |
1 |
25/60 |
50 |
|
Demographic Survey (Attachment I)
|
120 |
1 |
25/60 |
50 |
|
Focus group/interview (Attachment J) |
44 |
1 |
2 |
88 |
|
Focus group/interview (Attachment K) |
64 |
1 |
2 |
128 |
|
Focus group/interview (Attachment L) |
12 |
1 |
2 |
||
Total |
441 |
A.12.b) Annualized Burden Cost
The annualized cost to the respondent is presented in Table 4. Estimated Annualized Burden Costs. The total respondent cost is based on $25.72/hour which is the average hourly wage for all occupations according to the May 2019 National Occupational Employment and Wage Estimates from the U.S. Bureau of Labor Statistics.
Table 4. Estimated Annualized Burden Costs
Type of Respondents |
Data Collection |
Number of Respondents |
Number of Responses per Respondent |
Average Burden per Response (in hours) |
Average Hourly Wage Rate (in dollars) |
|
Total Respondent Cost |
Adults 18 years or older affected by the opioid epidemic (e.g., parents /caregivers of AI/AN children, Tribal Elders) living in Tribal urban and rural/ reservation communities
|
Information Letter (Attachment C) |
160 |
1 |
5/60 |
$25.72 |
|
$342.93 |
Telephone screening (Attachment D) |
160 |
1 |
25/60 |
$25.72 |
|
$1,714.67 |
|
Confirmation email/letter (Attachment E) |
120 |
1 |
5/60 |
$25.72 |
|
$257.20 |
|
Reminder email (Attachment G) |
120 |
1 |
5/60 |
$25.72 |
|
$257.20 |
|
Informed Consent (Attachment F)
|
120 |
1 |
25/60 |
$25.72 |
|
$1,286.00 |
|
Demographic Survey (Attachment I)
|
120 |
1 |
25/60 |
$25.72 |
|
$1,286.00 |
|
Focus group/interview (Attachment J) |
44 |
1 |
2 |
$25.72 |
|
$2,263.36 |
|
Focus group/interview (Attachment K) |
64 |
1 |
2 |
$25.72 |
|
$3,292.12 |
|
Focus group/interview (Attachment L) |
12 |
1 |
2 |
$25.72 |
|
$617.28 |
|
Total |
|
|
$11,316.76 |
A.13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
There will be no direct costs to the respondents other than their time to participate in the data collection. CDC does not anticipate providing capital or start up or other related costs to private entities.
A.14. Annualized Cost to the Government
Governmental costs for the ICR include personnel costs for federal staff overseeing the project, data collection instrument and OMB materials development, and data collection, analysis, and reporting. There are no equipment or overhead costs; however, a contractor is being used to support this effort. The costs for contract labor hours include planning and design, development of study protocols, recruitment of respondents, data preparation, data analysis, report writing, and dissemination of findings. The contract amount to plan, conduct, and analyze the data is $636,980. Thus, the total cost to the government for the project is $651,396 (see Table 5. Estimated Annualized Cost to the Government for the breakdown of the governmental costs for this ICR).
Table 5. Estimated Annualized Cost to the Government
Labor |
Cost |
CDC personnel for project oversight (15% GS-13 scientist)
|
$14,417
|
Contract labor for planning and design, development of study protocols, recruitment of respondents, data collection, data preparation, data analysis, report writing, and dissemination of findings
|
$636,980
|
Total estimated government costs |
$651,396 |
A.15. Explanation for Program Changes or Adjustments
This is a new information collection.
A.16. Plans for Tabulation and Publication, and Project Time Schedule
The schedule for data collection, analysis, and reporting is shown in Table 6. Project Time Schedule.
Data will be stored in password-protected files on a secure server. The research team will utilize qualitative analysis software to organize and code the focus group/interview transcripts. A preliminary list of codes along with a short codebook that describes the definition, inclusion and exclusion criteria, examples of code application, and source of each code. The research team will code a small subset of the qualitative data to identify emergent key themes and then refine and update the codebook to include any new codes. The research team will analyze the qualitative data to identify patterns and themes related to the aims of the research and summarize findings and analyze qualitative data from the survey using descriptive statistics. The research team may draw comparisons based on respondent group (e.g., Tribal Leaders) and/or Tribal community (e.g., rural/reservation). The research team will prepare site-specific summaries using standardized templates with possible adaptations based on site preferences to document key findings. All data will be reported in aggregate form. Study results will be shared with the Tribal communities and other federal agencies. Dissemination could include manuscript(s) in peer-reviewed journals, conference presentations, briefs, webinars, and other written products.
Table 6. Project Time Schedule
Activity |
Timeline from Contract Award |
Conduct Focus Group and/or Interviews |
Within 1 – 24months of OMB approval
|
Focus Groups or Interview Summary Reports, Related Notes, Audio Recordings, and Debrief Documents |
Within 18 months of OMB approval |
Codebook and Themes |
Within 18 months of OMB approval |
Methodology, Data Analyses, Summary Reliability Analysis, and Results Sections |
Within 20 months of OMB approval |
Summary of Tribal Preferences and Requirements for Reporting |
Within 20 months of OMB approval |
Manuscript of Research Results and Timeline Final Manuscript(s) |
Within 24 months of OMB approval |
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate
We are not requesting an exemption. The display of the OMB expiration date is not inappropriate.
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.
References
Anda, R. F., Brown, D. W., Felitti, V. J., Dube, S. R., & Giles, W. H. (2008). Adverse childhood experiences and prescription drug use in a cohort study of adult HMO patients. BMC Public Health, 8, 1-9.
Antone, C. F., Miguel Kahn-John, M., & Arambula Solomon, T. G. (2016). Native American Cultural Wisdon Declaration. Retrieved from http://www.ncai.org/resolutions/SPO-16-016_resolution.pdf
Beebe, T. J., Davern, M. E., McAlpine, D. D., Call, K. T., Rockwood, T. H. (2005). Increasing response rates in a survey of Medicaid enrolless: the effect of a prepaid monetary incentive and mixed modes (mail and telephone). Medical Care, 43, 411-414.
Biden, J. (2021). White House Memorandum on Tribal Consultation and Strengthening Nation-
to-Nation Relationships. https://www.whitehouse.gov/briefing-room/presidential-actions/2021/01/26/memorandum-on-tribal-consultation-and-strengthening-nation-to-nation-relationships/
Birnbaum, H. G., White, A. G., Schiller, M., Waldman, T., Cleveland, J. M., & Roland, C. L. (2011). Societal costs of prescription opioid abuse, dependence, and misuse in the United States. Pain Med, 12, 657-667. doi:10.1111/j.1526-4637.2011.01075.x
Breiding, M. J., Chen, J., & Black, M. C. (2014). Intimte partner violence in the United States - 2010. Atlanta, GA: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention.
Brockie, T., Dana-Sacco, G., Wallen, G., Wilcox, H., & Campbell, J. (2015). The relationship of adverse childhood experiences to PTSD, depression, poly-drug use and suicide attempt in reservation-based Native American adolescents and young adult. American Journal of Community Psychology, 55, 411-421.
Centers for Disease Control and Prevention (2013). CDC/ATSDR Tribal Consultation Policy (Updated 11/12/2013).
Centers for Disease Control and Prevention (2019). Preventing Adverse Childhood Experiences:
Leveraging the Best Available Evidence. Atlanta, GA: National Center for Injury
Prevention and Control, Centers for Disease Control and Prevention.
Centers for Disease Control and Prevention (2016). Preventing Multiple Forms of Violence: A Strategic Vision for Connecting the Dots. Atlanta, GA: Division of Violence Prevention, National Center for Injury Prevention and Control, Centers for Disease Control and Prevention.
Charles, L. W., Robert, F. A., Shanta, R. D., & Vincent, J. F. (2003). Violent childhood experiences and the risk of intimate partner violence in adults: Assessment in a large health maintenance organization. Journal of Interpersonal Violence, 18, 166-185. doi:10.1177/0886260502238733
Derefinko, K. J., Salgado Garcia, F. I., Talley, K. M., Bursac, Z., Johnson, K. C., Murphy, J. G., McDevitt-Murphy, M. E., Andrasik, F., & Sumrok, D. D. (2019). Adverse childhood experiences predict opioid relapse during treatment among rural adults. Addictive Behavior, 96, 171-174.
Derogatis, L. R. (2001). Brief Symptom Inventory (BSI)-18: Administration, scoring and procedures manual. Minneapolis, MN: NCS Pearson.
Dube, S. R., Anda, R. F., Felitti, V. J., Chapman, D. P., Williamson, D. F., & Giles, W. H. (2001). Childhood abuse, household dysfunction, and the risk of attempted suicide throughout the life span: Findings from the Adverse Childhood Experiences Study. JAMA, 286, 3089-3096.
Dykema, J., Stevenson, J., Kniss, C., et al. (2012). Use of monetary and nonmonetary incentives to increase response rates among African Americans in the Wisconsis Pregnancy Risk Assessment Monitoring System. Maternal Child Health Journal, 16, 785-791.
Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., & Kwan, I. (2002).
Increasing response rates to postal questionnaires: Systematic review. BMJ: British Medical Journal, 324(7347), 1183–1185.
Elm, J. H. L. (2020). Adverse Childhood Experiences and Internalizing Symptoms Among
American Indian Adults with Type 2 Diabetes. Journal of Racial and Ethnic Health Disparities. 7(5): 958-966.
Elm, J. H., Lewis, J. P., Walters, K. L., & Self, J. M. (2016). "I'm in this world for a reason":
Resilience and recovery among American Indian and Alaska Native two-spirit women. J Lesbian Stud, 20(3-4), 352-371. https://doi.org/10.1080/10894160.2016.1152813.
Elm, J. H. L., Walls, M. L., & Aronson, B. D. (2019). Sources of Stress Among Midwest
American Indian Adults with Type 2 Diabetes. Am Indian Alsk Native Ment Health Res, 26(1), 33-62. https://doi.org/10.5820/aian.2601.2019.33.
Exec. Order No. 13175, Fed. Reg. Vol. 65, No. 218 (November 6, 2000) https://www.federalregister.gov/d/00-29003
Feder, K. A., Mojtabai, R., Musci, R. J., & Letourneau, E. J. (2018). U.S. adults with opioid use
disorder living with children: Treatment uses and barriers to care. Journal of Substance
Abuse Treatment, 93, 31-37.
Felitti, V. J., Anda, R. F., Nordenberg, D., Williamson, D. F., Spitz, A. M., Edwards, V., . . . Marks, J. S. (1998). Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. The Adverse Childhood Experiences (ACE) Study. Am J Prev Med, 14, 245-258.
Finkelhor, D., Shattuck, A., Turner, H., & Hamby, S. (2015). A revised inventory of adverse childhood experiences. Child Abuse and Neglect, 48, 13-21.
Gilbert, L. K., Breiding, M. J., Merrick, M. T., Thompson, W. W., Ford, D. C., Dhingra, S. S., & Parks, S. E. (2015). Childhood adversity and adult chronic disease: An update from ten states and the District of Columbia, 2010. Am J Prev Med, 48, 345-349. doi:10.1016/j.amepre.2014.09.006
Grayshield, L., Rutherford, J., Salazar, S., Mihecoby, A., & Luna, L. (2015). Understanding and healing historical trauma: The perspectives of Native American elders. Journal of Mental Health Counseling, 37, 295-307.
Hamby, S., Schultz, K., & Elm, J. (2020). Understanding the burden of trauma and victimization among American Indian and Alaska native elders: Historical trauma as an element of poly-victimization. Journal of Trauma and Dissociation, 21, 172-186.
Henson, M., Sabo, S., Trujillo, A., & Teufel-Shone, N. (2017). Identifying Protective Factors to Promote Health in American Indian and Alaska Native Adolescents: A Literature Review. Journal of Primary Prevention, 38, 5-26.
Jones, M. S., Worthen, M. G. F., Sharp, S. F., & McLeod, D. A. (2020). Native American and
Non-Native American women prisoners: Adverse childhood experiences, and the
perpetration of physical violence in adult intimate relationships. Journal of Interpersonal Violence, Published Online.
Kenny, M., & Singh, G. (2016). Adverse childhood experiences among American Indian/Alaska Native children: The 2011-2012 national survey of children’s health. Scientifica, Vol. 2016, article 7424239.
Merrick, M. T., Ford, D. C., Ports, K. A., Guinn, A. S., Chen, J., Klevens, J., Metzler, M., Jones,
C. M., Simon, T. R., Daniel, V. M., Ottley, P., Mercy, J. A. (2019). Vital Signs: Estimated proportion of adult health problems attributable to adverse childhood experiences and implications for prevention - 25 states, 2015-2017. MMWR Morbidity and Mortality Weekly Report, 68, 999-1005.
Mersky, J., Janczewski, C., & Nitkowski, J. (2018). Poor mental health among low-income
women in the US: The roles of adverse childhood and adult experiences. Social Science & Medicine, 206, 14-21
Meyer, R., Patel, A. M., Rattana, S. K., Quock, T. P., & Mody, S. H. (2014). Prescription opioid abuse: A literature review of the clinical and economic burden in the United States. Popul Health Manag, 17, 372-387. doi:10.1089/pop.2013.0098
National Child Traumatic Stress Network. (2019). Retrieved from https://www.nctsn.org/what-is-child-trauma/about-child-trauma.
Narayan, A.I., Rivera, L. M., Bernstein, R. E., Harris, W. W., & Lieberman, A. F. (2018). Positive childhood experiences predict less psychopathology and stress in pregnant women with childhood adversity: A pilot study of the benevolent childhood experiences (BCEs) scale. Child Abuse and Neglect, 78, 19-30.
Ports, K. A., Ford, D. C., & Merrick, M. T. (2016). Adverse childhood experiences and sexual victimization in adulthood. Child Abuse and Neglect, 51, 313-322. doi:10.1016/j.chiabu.2015.08.017
Romanowicz, M., Vande Voort, J. L., Shekunov, J., Oesterle, T. S., Thusius, N. J., Rummans, T. A., Croarkin, P. E., Karpyak, V. M., Lynch, B. A., & Schak, K. M. (2019). The effects of parental opioid use on the parent-child relationship and children's developmental and behavioral outcomes: A systematic review of published reports. Child and Adolescent Psychiatry and Mental Health, 13, Article 5.
Sapp, A. J., & Hooten, P. (2019). Working with families impacted by the opiod crisis: Education, best practices, and providing hope. Archives of Psychiatric Nursing, 33, 3-8.
Schofield, T. J., Lee, R. D., & Merrick, M. T. (2013). Safe, stable, nurturing relationships as a moderator of intergenerational continuity of child maltreatment: a meta-analysis. Journal of Adolescent Health, 53 (4 Suppl), S32-38. doi:10.1016/j.jadohealth.2013.05.004
Shaghai, A., Bhopal, R. S., & Sheikh, A. (2011). Approaches to recruiting 'hard-to-reach'
populations into research: A review of the literature. Health Promotion Perspective, 1, 86-94.
Singer, E., van Hoewyk, J., & Maher, M. P. (2000). Experiments with incentives in telephone
surveys. Public Opinion Quarterly, 64, 171-188.
Stein, M. D., Conti, M. T., Kenney, S., Anderson, B. J., Flori, J. N, Risi, M. M., & Bailey, G. L.
(2017). Adverse childhood experience effects on opioid use initiation, injection drug use,
and among persons with opioid use disorder. Drug and Alcohol Dependence, 179, 325-329.
Tipps, R. T., Buzzard, G. T., & McDougall, J. A. (2018). The opioid epidemic in Indian Country. Journal of Law, Medicine & Ethics, 46, 422-436.
Walls, M., Sittner, K. J., Whitbeck, L. B., Herman, K., Gonzalez, M., Elm, J. H., ... & Hoyt, D. R. (2020). Prevalence of mental disorders from adolescence through early adulthood in American Indian and First Nations communities. International Journal of Mental Health and Addiction, 1-15.
Warne, D., Dulacki, K., Spurlock, M., Meath, T., Davis, M. M., Wright, B., & McConnell, K. J. (2017). Adverse childhood experiences (ACE) among American Indians in South Dakota and associations with mental health conditions, alcohol use, and smoking. Journal of Health Care for thePpoor and Underserved, 28(4), 1559-1577.
Wright, M. O. D., Masten, A. S., & Narayan, A. J. . (2013). Resilience processes in development: Four waves of research on positive adaptation in the context of adversity. In Handbook of Resilience in Children (pp. 15-37). Boston, MA.: Springer.
Whitebeck, L. B., Adams, G. W., Hoyt, D. R., & Chen, X. (2004). Conceptualizing and measuring historical trauma among American Indian people. American Journal of Community Psychology, 33, 119-130.
Whitesell, N. R., Beals, J., Crow, C. B., Mitchell, C. M., & Novins, D. L. (2012). Epidemiology and etiology of substance use among American Indians and Alaska Natives: Risk, protection, and implications for prevention. The American Journal of Drug and Alcohol Abuse. 38, 376-382.
Whitesell, N. R., Sarche, M., Keane, E., Mousseau, A. C., & Kaufman, C. E. (2018). Advancing scientific methods in community and cultural context to promote health equity: Lessons from intervention outcomes research with American Indian and Alaska Native communities. American Journal of Evaluation, 39, 42-57.
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
File Title | OMB Application for |
Author | Shortt, Joann Wu (CDC/DDNID/NCIPC/DVP) |
File Modified | 0000-00-00 |
File Created | 2022-05-27 |