Ssa_17amp_030718

SUPPORTING STATEMENT: PART A

Evaluation of the SAMHSA Naloxone Education and Distribution Program

OMB# 0920-XXX

March 3, 2017

Point of Contact:

Sally Thigpen

Contact Information:

Centers for Disease Control and Prevention

National Center for Injury Prevention and Control

4770 Buford Highway NE MS F-64

Atlanta, GA 30341-3724

Phone: 770-488-3892

Fax: 770-488-3551

Email: sti9@cdc.gov

CONTENTS

Section Page

A. Summary table 3

Justification 3

A.1. Circumstances Making the Collection of Information Necessary 3

A.2. Purpose and Use of Information Collection 5

A.3. Use of Improved Information Technology and Burden Reduction 8

A.4. Efforts to Identify Duplication and Use of Similar Information 8

A.5. Impact on Small Businesses or Other Small Entities 9

A.6. Consequences of Collecting the Information Less Frequently 9

A.7. Special Circumstances Relating to the Guidelines of

5 CFR 1320.5(d)2 9

A.8. Comments in Response to the Federal Register Notice and

Efforts to Consult Outside the Agency 10

A.9. Explanation of Any Payment or Gift to Respondents 9

A.10. Protection of the Privacy and Confidentiality of Information Provided by Respondents 10

A.11. Institutional Review Board (IRB) and Justification for Sensitive Questions 11

A.12. Estimates of Annualized Burden Hours and Costs 11

A.13. Estimates of Other Total Annual Cost Burden to Respondents

or Record Keepers 14

A.14. Annualized Cost to the Government 14

A.15. Explanation for Program Changes or Adjustments 134

A.16. Plans for Tabulation and Publication and Project Time Schedule 14

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate 15

A.18. Exceptions to Certification for Paperwork Reducation Act

Submissions 15

References 16

Attachments

A Authorizing Legislation: Public Health Service Act

B Published 60-Day Federal Register Notice

C IRB Documentation

D Focus Group Guides

E Key Informant Interview Guides

E1. Key Informant Interview Guide-Grantee

E2. Key Informant Interview Guide-Partner

E3. Key Informant Interview Guide-Layperson

F Recruitment Contact scripts and informed consent

G Key Informant Selection Tool

H Memorandum of Understanding – CDC/SAMHSA

SUMMARY TABLE

• Goal of the study: To evaluate the Substance Abuse and Mental Health Services Administration (SAMHSA) Grants to Prevent Prescription Drug/Opioid Overdose-Related Deaths (the PDO/naloxone grant). This evaluation will seek to describe and understand the scope and outcomes of the SAMHSA PDO/naloxone grant. Through this grant, SAMHSA awarded funding to 12 states. The funding is aimed at reducing the number of prescription drug/opioid overdose (PDO)-related deaths and adverse events among individuals 18 years of age and older through educating and training first responders and other key community sectors on the prevention of PDO-related deaths, including the purchase and distribution of naloxone.

• Intended use of the resulting data: By following outcomes over time with qualitative measures, CDC will increase the understanding of the outcomes of the PDO/naloxone grant on overdose fatalities and how program effectiveness may vary among different sub-populations and settings, and will increase knowledge of barriers and facilitators to program implementation.

• Methods to be used to collect: Key informant interviews and focus groups.

• The subpopulation to be studied: Participants in the activities enacted by the twelve state grant recipients. This will include state administrators of the grant and other PDO/Naloxone grant stakeholders including advisory council members, first responders, social service providers, and laypersons including end users and their family and friends.

• How data will be analyzed: All focus groups and interviews will be analyzed through qualitative content analysis, including utilization of a systematic coding scheme and NVivo software.

A. JUSTIFICATION

A.1. Circumstances Making the Collection of Information Necessary

The Centers for Disease Control and Prevention (CDC) requests Office of Management and Budget (OMB) approval for 3 years for this new data collection and expects to request an extension for the remaining 2 years of the 5 year SAMHSA Grants to Prevent Prescription Drug/Opioid Overdose-Related Deaths (PDO/naloxone grant). SAMHSA is funding the grant and collecting grantee performance and progress data under OMB 0930-0354 with expiration 7/31/2020. CDC is responsible for conducting an evaluation of the PDO/naloxone grant as described on page 302, paragraph 2 of the Centers for Disease Control and Prevention’s FY 2016 Congressional Justification (https://www.cdc.gov/budget/documents/fy2016/fy-2016-cdc-congressional-justification.pdf).

The proposed information collection is authorized by the Public Health Services Act (PHS Act) which provides the legislative means for states to advance public health across the lifespan and to reduce health disparities. Section 301 (a) of the PHS Act, 42 U.S.C. 241 (a), authorizes grants to aid “other appropriate public authorities, scientific institutions, and scientists in the conduct of, and promote the coordination of, research, investigations, experiments, demonstrations, and studies relating to the cause, diagnosis, treatment, control and prevention of physical and mental diseases and impairments of man” (Attachment A).

Overdose deaths involving prescription opioids and heroin have reached crisis levels in the U.S. and continue to rise. In 2015, 591,000 Americans had a heroin use disorder and 2 million Americans had a prescription opioid use disorder (Center for Behavioral Health Statistics and Quality, 2016). From 2014-2015, overdose deaths from commonly prescribed opioids, heroin, and non-methadone synthetics increased 3%, 21%, and 72% respectively (Rudd, 2016A).

In 2014, over 2 million Americans either abused or were dependent on opioids, with 1.9 million Americans having a substance use disorder involving prescription pain relievers and 586,000 having a substance use disorder involving heroin (Substance Abuse and Mental Health Services Administration, 2015). Over only a 1-year period from 2013 to 2014, the death rate from the most commonly prescribed opioid pain relievers (natural and semisynthetic opioids) increased 9%, the death rate from heroin increased 26%, and the death rate from synthetic opioids (illicitly manufactured fentanyl and synthetic opioid pain relievers other than methadone) increased 80% (Rudd, 2016B). These figures translate to 115 people in the US dying of opioid overdoses every day (Centers for Disease Control and Prevention, 2016).

To combat the intensifying epidemic of opioid overdose deaths, evidence-based strategies need to be widely disseminated and implemented. Increasing access to naloxone, an opioid antagonist medication, is a compelling strategy because naloxone can reverse the respiratory depressive effects of an opioid overdose and be life-saving if administered in time. Naloxone has been used since the 1970s in hospitals and by emergency medical services (EMS) to reverse opioid overdoses, and studies have shown naloxone to be safe and effective at reducing opioid overdose deaths and cost-effective (Coffin, 2013; Walley, 2013).

To address the PDO crisis, the federal government has recently allocated funding to improve access to treatment and recovery services for opioid use disorders, reduce opioid related deaths, and strengthen drug misuse prevention efforts. One program resulting from the federal government’s efforts to address the opioid crisis is the SAMHSA Grants to Prevent Prescription Drug/Opioid Overdose-Related Deaths (PDO/naloxone grant). Through this program, SAMHSA awarded $10,345,984 to 12 states for naloxone education and distribution: Alaska, Arkansas, Illinois, Missouri, New Jersey, New Mexico, Oklahoma, South Carolina, Washington, Wisconsin, West Virginia, and Wyoming. The funding is aimed at reducing the number of PDO-related deaths and adverse events among individuals 18 years of age and older through educating and training first responders and other key community sectors on the prevention of PDO-related deaths, including the purchase and distribution of naloxone.

Consistent with the critical role that CDC plays in implementing and evaluating public health interventions and in alignment with HHS’s FY 2016 opioid plan, CDC’s Congressional Justification for FY 2016 included a request for funding to evaluate SAMHSA’s proposed PDO/naloxone grant program. Once funds were appropriated, CDC coordinated with HHS and SAMHSA to ensure evaluation plans were complementary and comprehensive. Through a memorandum of understanding between CDC and SAMHSA (Attachment H), CDC, with the support of our contractor Battelle, will evaluate the grant program. This evaluation will seek to describe and understand the scope and outcome of the program on overdose fatalities. It also seeks to increase the understanding of how program effectiveness may vary among different sub-populations and settings, and increase the knowledge of barriers and facilitators to program implementation. This information will be crucial in helping CDC and SAMHSA address the PDO epidemic and assess the effectiveness of the program on preventing and reducing the number of PDO-related fatalities and adverse events so that program improvements can be made and future funding mechanisms can utilize the most effective program elements and address any challenges that were experienced.

A.2. Purpose and Use of Information Collection

The purpose of the program mixed-methods evaluation is to help CDC and SAMHSA understand the implementation factors that influence the effectiveness of the PDO/naloxone grant on ameliorating the PDO epidemic and reducing the number of overdose fatalities and adverse events. The results will enable CDC and SAMHSA to understand and address any challenges and barriers to program implementation and inform continuous quality improvements during the funding cycle. SAMHSA will be collecting program monitoring and performance data. CDC will collect qualitative data from SAMHSA grantees and their partners and those who benefit from related activities. While the SAMHSA performance monitoring reporting system (OMB 0930-0354) tracks activities and collects associated administrative and performance related data, the qualitative data collection component of the program evaluation addressed in this ICR will answer questions related to contextual factors that can further explain why and how the grantees were able to achieve successful reversal and decreased mortality from naloxone administration. It will also provide insight into why some efforts were not as successful. Without this qualitative data collection, many of our key evaluation questions that can inform future efforts in this critical area of the opioid response will be left unanswered.

Table 1 below specifies the overarching evaluation questions of the mixed method design and more specific outcome measures aligned with the sources of information from the data collection efforts of CDC and SAMHSA. This shows how the qualitative data collection proposed here will compliment other data sources (i.e., grantee reporting as collected by SAMHSA OMB 0930-0354 and other reporting data from the state). Where qualitative data collection is collected in addition to one of these other sources of data, the qualitative data will provide rich detail that can only be collected through interviews and focus groups which allow the data collector to probe for specific details related to the outcome measure.

Table 1. Data Collection Instruments Aligned to CDC Evaluation

Evaluation Questions and Outcome Measures	Data Source
Did the PDO/naloxone grant lower prescription drug/opioid overdose-related morbidity and mortality in funded states?
1.1 What were the changes in morbidity/mortality related to prescription drug/opioid overdose?	SAMHSA OMB 0930-0354
1.2 What were the changes in prescription drug/opioid overdose reversals?	SAMHSA OMB 0930-0354
How did the PDO/naloxone grant change access to naloxone in funded states?
2.1 What were the changes in the availability of naloxone among high-need communities and partner organizations?	SAMHSA OMB 0930-0354
2.2 To whom was naloxone distributed and how?	FG, KII-G, KII-P, KII-L
2.3 Was naloxone distributed as intended? What factors facilitated or impeded distribution?	FG, KII-G
Did the PDO/naloxone funding increase infrastructure and sustainability for PDO prevention efforts in funded states?
3.1 What approaches were used to identify and engage partners? Which approaches were most successful?	FG, KII-G, KII-P
3.2 What worked well in forming the advisory council? What could have been done more effectively?	FG, KII-G
3.3. What approaches to developing systems for tracking outcomes were most successful or least successful and why?	FG, KII-G, KII-P
3.4 What barriers or facilitators were experienced during the planning, capacity building, or initial implementation stage and how were these addressed?	FG, KII-G
3.5 What approaches were taken to increase sustainability of this work?	FG, KII-G, KII-P
What were the ways that grantees provided naloxone training and education and to what audiences?
4.1 In what ways did education/training vary among different audiences?	KII-G, KII-P, KII-L
4.2 What were the barriers and facilitators to providing education and training and how have they been addressed?	FG, KII-G
4.3 What approaches to training and education worked best? Which were least effective?	FG, KII-G, KII-P, KII-L
4.4 In what ways was naloxone education/distribution being integrated with other substance abuse and/or treatment programs?	KII-G, KII-P
4.5 What were attitudes toward use of naloxone?	FG, KII-G, KII-P, KII-L
What factors facilitated or impeded program implementation and why?	KII-G, KII-P
Did the PDO/naloxone funding increase PDO prevention-related collaboration and partnerships?
6.1 In what ways did grantees collaborate and coordinate funding streams?	SAMHSA OMB 0930-0354
6.2 What ways of collaborating and coordinating funding streams were most successful and why?	FG
6.3 What new strategic alliances were formed?	SAMHSA OMB 0930-0354
6.4 What were the barriers and facilitators to working with partners?	FG, KII-G, KII-P
What approaches to address behavioral health disparities were most successful and why?
7.2 What approaches to decreasing differences in access, service and outcomes among disparate populations were most successful and why?	FG, KII-G, KII-P, KII-L
How did state/local support for naloxone use change distribution and use?	FG, KII-G, KII-P, KII-L
How did varying state & local policies/laws change naloxone distribution and use?	FG, KII-G, KII-P, KII-L
What program elements were most successful and why? Which were least successful?
10.1 What were key program accomplishments?	KII-G, KII-P
10.2 What program aspects worked best or were least effective?	FG, KII-G, KII-P
10.3 In what ways could future versions of PDO/naloxone funding be improved?	FG, KII-G, KII-P
10.4 In what ways has stakeholder support for naloxone changed?	FG, KII-G, KII-P, KII-L

FG: Focus Group Guide (Attachment D)

KII-G: KII Guide-Grantee (Attachment E1)

KII-P: KII Guide-Partner (Attachment E2)

KII-L: KII Guide-Layperson (Attachment E3)

To answer these evaluation questions, data will be collected annually through key informant interviews and focus groups with grantees and select stakeholders at each of the 12 SAMHSA-funded sites. Table 1 above shows which data collection instruments address each evaluation question and outcome measure. The data collection will include telephone and in-person interviews conducted annually with respondents. Respondents for the data collection will include grantee program staff, and key stakeholders to include, but not be limited to, state and local health department officials, PDO Advisory Council members, first responders, as well as naloxone end-users and their friends or families. Data will be collected annually in order to be responsive to any barriers or challenges that might arise so that ongoing program improvements or refinements can be made.

A.3. Use of Improved Information Technology and Burden Reduction

The program evaluation will be collecting measures about the implementation and outcomes of the PDO/naloxone grant. The interviews will collect qualitative data that are not amenable to collection through a survey format. The data for the study will be collected through key informant interviews and focus groups conducted either by telephone or in person. These methods will most effectively allow us to query about certain program outcomes, and barriers and facilitators to implementation.

To reduce burden on the respondent, data collection staff will travel to the site to meet with interviewees and/or schedule a telephone interview at a time that is convenient for the respondent. Also, conduct of the focus groups over the telephone reduces burden to participants since they do not have to travel to a meeting location. The focus group and key informant interview guides were designed to collect the minimum information necessary for the purposes of this project. Key informant discussion guides are tailored to a specific audience (i.e. grantees, partners, laypersons) so that we are collecting specific information only from those most able to provide that information.

A.4. Efforts to Identify Duplication and Use of Similar Information

This program evaluation is a new initiative. The funding the grantees received is from FOA No. SP-16-005 titled Grants to Prevent Prescription Drug/Opioid Overdose-Related Deaths (Short Title: PDO grant). SAMHSA is funding the grant and CDC is responsible for conducting the evaluation. In order to coordinate evaluation efforts and avoid duplication, CDC has worked collaboratively with colleagues at SAMHSA to design the study. As part of the evaluation, CDC and contractor Battelle will collect qualitative data from sites, while SAMHSA will collect measures from grantees as part of the grant requirements. Battelle and CDC will utilize the Grantee Progress Reports provided to SAMHSA and available secondary sources such as state level data to inform this evaluation as much as possible. The qualitative data collection we are describing in this OMB package is not available elsewhere. Qualitative evaluation tools have been specifically designed for this study to obtain detailed information to inform the evaluation questions and outcome measures. CDC and SAMHSA have consulted with each other throughout the evaluation design to make sure the data and data collection efforts are complementary and not duplicative.

A.5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection.

A.6. Consequences of Collecting the Information Less Frequently

Focus group and interview data collection for this study will occur annually for 4 years. The purpose of annual data collection is to gather information which will identify facilitators and barriers at various stages of program planning and implementation. Annual data collection allows us to provide evaluation feedback to CDC and SAMHSA for continuous quality improvement and to inform course corrections over the course of the 5-year grant period.

The consequence of not collecting the information would be to limit CDC’s and SAMHSA’s efforts to assess the outcomes of their efforts in context of implementation factors that may influence these outcomes. Without the data collection activities proposed, CDC and SAMHSA would not have essential information necessary to provide support to current programs and make critical decisions about the future support of this program.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

The request fully complies with the regulation 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A.8.a) Federal Register Notice

A 60-day Federal Register Notice was published in the Federal Register on July, 17, 2017, vol. 82, No. 135, pp. 32704 (Attachment B). There were no comments to the 60-day Federal Register Notice.

A.8.b) Efforts to Consult Outside the Agency

In addition to consulting with federal colleagues at SAMHSA on the development of the evaluation design CDC has also consulted with funded grantees to further inform evaluation planning and data collection activities from a participatory perspective. The following grantees were consulted in this effort:

Bridget L. Hanson, PhD, University of Alaska Anchorage

Michelle Nienhius, South Carolina Department of Alcohol and Other Drug Abuse Services

Elizabeth Collier, Wisconsin Department of Health

James Matney, West Virginia Bureau for Behavioral Health and Health Facilities

Rachel Winograd, University of Missouri, St. Louis

Drew Dobro, Wyoming Public Health Division

Caleb Banta-Green, University of Washington

Liz Lilliot, Pacific Institute of Research and Evaluation

A.9. Explanation of Any Payment or Gift to Respondents

Layperson key informants including persons who inject drugs will receive gift card incentives for their participation in the study. They will be offered a $25 gift card as a token of appreciation for their participation in the in-person interviews. This evaluation faces challenges in achieving the needed response rate due to the nature of illegal drug use, also this population is particularly difficult to include as study participants. It has been found that utilizing incentives with this population increases participation as well as satisfaction with the study (Slomka, 2007; Topp, 2013; Festinger, 2008; Desmond, 1995).

Without addressing the response rate challenge, the resulting low-response can produce non-response basis if non-respondents are different from respondents on characteristics and outcomes of interest to the evaluation. Fortunately, these challenges can be reduced by offering tokens of appreciation to respondents for participating in the interviews (James 1997, Singer & Couper 2008). Further, recent studies show that incentives are effective in increasing response rates. A recent study of four alcohol and tobacco use studies showed that offering incentives increased on-time survey completion from 18% to 68% and using incentives achieved response rates over 90% (Smith et. al., 2017). A longitudinal health study of individuals impacted by the 9/11 terrorist attacks in New York City found that offering a $10 incentive increased survey returns by 30% and increased the overall response rate by 18% (Yu et. al., 2017).

A.10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

This submission has been reviewed by the NCIPC’s Information Systems Security Officer, who has determined that the Privacy Act does not apply. Personal information will not be collected or stored as part of the data collection efforts. Information in identifiable form will not be requested as part of the individual interview, focus group or state evaluation plan template data collection. Although contact information is obtained, the contact person provides information about the organization, not personal information. Laypersons contact information will come from secondary data from already-established records of a convenience sample of laypersons that participate in naloxone education and distribution activities that are conducted by grantees as part of the PDO grant.

Participants will not be identified by name or description in any reports. The data collection instruments and protocols will be reviewed by the contractor’s Institutional Review Board. All electronic data will be maintained on password protected computers on a password protected network. Access will be limited to authorized study personnel who have need for the data for collection or analysis. Any hard copy data will be kept in locked file cabinets in offices with access limited to those staff working on the study. Data will be stored in accordance with Battelle security and retention requirements. Through a study consent process, participants will be made aware of confidentiality and protection of their information as well as the limitations of this. Consent material and email/phone communications make clear that participation in interviews and focus groups is completely voluntary.

A.11. Institutional Review Board (IRB) and Justification for Sensitive Questions

IRB Approval

CDC has received IRB approval through Battelle IRB (Attachment C).

Sensitive Questions

Due to the content of the PDO grant activities, which includes education on and distribution of naloxone, a drug given to reverse overdose, individual interviews and focus groups may discuss drug use and use of naloxone. This is necessary to evaluate the effectiveness and implementation of the grant activities. Participants will be fully informed of applicable privacy safeguards to help address any concerns about inadvertent disclosure of sensitive information.

A.12. Estimates of Annualized Burden Hours and Costs

The total annualized burden hours are 381 as described in Table 2 below. We anticipate conducting 1 focus group per grantee state each year consisting of advisory committee members (12 focus groups) and 2 cross-site focus groups annually consisting of state grantee staff from across states (see Attachment D). This totals 14 focus groups of 10 participants each, using the same focus group discussion guide, for a total of 140 focus group participants. Three key informant individual interviews will be conducted annually with each of the 12 grantee states’ program staff totaling 36 interviews annually (see Attachments E1-E3). Seven individual interviews will be conducted with partner stakeholders annually in each of the 12 grantee states totaling 84 interviews annually. In addition, 2 interviews will be conducted annually in each of the 12 grantee states with naloxone end users totaling 24 interviews annually. To recruit interview participants we will utilize a very brief recruitment email (see Attachment F) and to assist with the identification of stakeholders for interviews and focus groups, grantee staff will be requested to complete the Key Informant Selection Tool annually (see Attachment G).

Table 2. Estimated Annualized Burden Hours

Type of Respondents	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)
PDO/Naloxone Advisory Committee Members and Grantees	Focus Group Discussion Guide (Att. D)	140	1	1.5	210
PDO/Naloxone Grantees	Key Informant Interview Guide for Grantees (Att. E1)	36	1	1	36
PDO/Naloxone Stakeholders and Partners	Key Informant Interview Guide for Partners (Att. E2)	84	1	1	84
PDO/Naloxone Laypersons	Key Informant Interview Guide for Laypersons (Att. E3)	24	1	1	24
All participants (PDO Naloxone grantees, advisory committee, stakeholders and partners, laypersons)	Recruitment contact script (Att. F)	284	1	5/60	24
PDO/Naloxone Grantees	Key Informant Selection Tool (Att. G)	12	1	15/60	3
Total					381

A.12.b) Annual burden cost

Table 3 below describes the annualized burden costs associated with data collection. Data will be collected from PDO/Naloxone grant stakeholders and partners, including advisory committee members. The median hourly wage per the Bureau of Labor Statistics is $17.40. State Grantee staff are in community and social service occupations. The median hourly wage for that group is $20.20, also per the Bureau of Labor Statistics. To reflect a more accurate burden cost for layperson, the calculation is based on a $50,000 annual salary per the November 2017 report of the Council of Economic Advisers titled, “The Underestimated Costs of the Opioid Crisis.”

Using this annual salary estimate, the hourly wage for layperson in the cost burden table is $24.04 per hour and was used to estimate costs for naloxone end users and other laypersons who will participate in individual interviews.

Table 3. Estimated Annualized Burden Costs

Type of Respondents	Form Name	Total Burden (in hours)	Average Hourly Wage Rate (in dollars)	Total Respondent Cost
PDO/Naloxone Advisory Committee Members	Focus Group Discussion Guide (Att. D)	180	$17.40	$3,132.00
PDO/Naloxone Grantees	Focus Group Discussion Guide (Att. D)	30	$20.20	$606.00
PDO/Naloxone Grantees	Key Informant Interview Guide for Grantees (Att. E1)	36	$20.20	$727.20
PDO/Naloxone Stakeholders and Partners	Key Informant Interview Guide for Partners (Att. E2)	84	$17.40	$1,461.60
PDO/Naloxone Laypersons	Key Informant Interview Guide for Laypersons (Att. E3)	24	$24.04	$576.96
PDO/Naloxone Stakeholders and Partners; Advisory Committee Members	Recruitment contact script (Att. F)	17	$17.40	$295.80
PDO Naloxone Grantees	Recruitment contact script (Att. F)	5	$20.20	$101.00
PDO/Naloxone Laypersons	Recruitment contact script (Att. F)	2	$24.04	$48.08
PDO/Naloxone Grantees	Key Informant Selection Tool (Att. G)	3	$20.20	$60.60
Total	$7,009.24

A.13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

None. There is no other total annual cost burden to respondents or record keepers resulting from this collection of information.

A.14. Annualized Cost to the Government

Table 4. Estimated Annualized Cost to the Government

Type of Cost	Description of Services	Annual Cost
GS-14 public health analyst at 10% FTE GS-13 program analyst at 10% FTE GS-13 program analyst at 10% FTE GS-12 contractors at 20% FTE		$12,127 $8,947 $7,895 $13,275
Contractor costs	Evaluation Contract	$523,607
Total Annual Estimated Costs		$565,851

A.15. Explanation for Program Changes or Adjustments

This is a new data collection.

A.16. Plans for Tabulation and Publication, and Project Time Schedule

All focus group and interview data will be analyzed through qualitative content analysis. To analyze the data, we will develop a systematic coding scheme and method of presentation that links to the assessment questions. The interview and focus group transcripts will be loaded into a QSR NVivo® database, which will support data management and analysis. Once data are entered, the transcripts can be coded according to common themes that emerge. Battelle researchers will develop a codebook keyed to the interview guides and evaluation objectives/questions. Each text file will be read carefully, and each segment of text coded with one or more codes. Codes will be used to classify text into categories related to each assessment question and important themes. To ensure coding trustworthiness, project team members will meet to discuss the process, finalize the codebook, and help interpret preliminary results. Because of potential differences between the selected programs, some of the themes or codes will not be applicable to all programs, while others will be used for all cases. This coding approach will allow us to identify both anticipated and emergent themes, accounting for all relevant categories found in the data, and will serve as the basis for within-case analysis and descriptions, while also facilitating multiple-case analyses. Analysis will be conducted and reported on annually.

Table 5. Proposed project timeline

Activity	Time Schedule
Conduct key informant interviews and focus groups	1-5 months after OMB approval; annually thereafter
Analyze data	6-7 months after OMB approval; annually thereafter
Final report and presentation of data	8-9 months after OMB approval; annually thereafter

All participating programs will receive a CDC-SAMHSA approved summary of the evaluation findings at the conclusion of the study. This summary can be used to articulate program outcomes to key stakeholders and is instrumental in quality improvement efforts.

This data collection is being conducted in response to the Achieving Public Health Impact Through Research Contract Mechanism (Contract No. HHSD2002013M53942), RFTOP Number 2016-00440 titled Evaluation of the SAMHSA Naloxone Education and Distribution Program. We will submit a revision to the ICR for the fourth year of data collection.

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

The display of the OMB expiration date is not inappropriate.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

REFERENCES

Center for Behavioral Health Statistics and Quality. Key substance use and mental health indicators in the United States: Results from the 2015 National Survey on Drug Use and Health (HHS Publication No. SMA 16-4984, NSDUH Series H-51). 2016. Retrieved from http://www.samhsa.gov/data/

Centers for Disease Control and Prevention. Drug overdose deaths in the United States continue to increase in 2016. https://www.cdc.gov/drugoverdose/epidemic/. Accessed July 29, 2016.

Centers for Disease Control and Prevention. CDC FY 2016 Congressional Justification (page 302, paragraph 2). https://www.cdc.gov/budget/documents/fy2016/fy-2016-cdc-congressional-justification.pdf

Coffin PO, Sullivan SD. Cost-effectiveness of distributing naloxone to heroin users for lay overdose reversal. Ann Intern Med. 2013. Jan 1;158(1):1-9.

Desmond DP, Maddux JF, Johnson TJ, Confer BA. Obtaining follow-up interviews for treatment evaluation. Journal of Substance Abuse Treatment. 1995. 12(2):95-102

Festinger, DS, Marlowe, DB, Dugosh, KL, Croft, JR, Arabia, PL. Higher magnitude cash payments improve research follow-up rates without increasing drug use or perceived coercion. Drug and Alcohol Dependence. 2008. 96(1-2):128-135. http://dx.doi.org/10.1016/j.drugalcdep.2008.02.007

James, T. L. Results of the wave I incentive experiment in the 1996 Survey of Income and Program Participation. Report to the U.S. Bureau of the Census, 1997. 834-839.

Rudd RA, Seth P, David F, Scholl L. Increases in Drug and Opioid-Involved Overdose Deaths — United States, 2010–2015. MMWR Morb Mortal Wkly Rep 2016 (A); 65:1445–1452.

Rudd RA, Aleshire N, Zibbell JE, Gladden RM. Increases in Drug and Opioid Overdose Deaths--United States, 2000-2014. MMWR Morb Mortal Wkly Rep. 2016 (B) Jan 1;64(50-51):1378-82.

Slomka J, McCurdy S, Ratliff EA, Timpson S, Williams ML. Perceptions of Financial Payment for Research Participation among African-American Drug Users in HIV Studies. Journal of General Internal Medicine. 2007. 22(10):1403-1409. doi:10.1007/s11606-007-0319-9.

Substance Abuse and Mental Health Services Administration, Center for Behavioral Health Statistics and Quality. Behavioral health trends in the United States: Results from the 2014 National Survey on Drug Use and Health. 2015. Rockville, MD: Substance Abuse and Mental Health Services Administration.

Smith, L. J., McNamara, P. J., & King, A. C. Optimizing follow-up and study retention in the 21st century: Advances from the front line in alcohol and tobacco research. Drug and Alcohol Dependence, 2017. 175:171-178.

Topp L, Islam MM, Day CA. Relative efficacy of cash versus vouchers in engaging opioid substitution treatment clients in survey-based research. Journal of Medical Ethics. 2013. 39:253-256.

Yu, S., Alper, H. E., Nguyen, A. M., Brackbill, R. M., Turner, L., Walker, D. J., ... & Zweig, K. C. The effectiveness of a monetary incentive offer on survey response rates and response completeness in a longitudinal study. BMC medical research methodology, 2017. 17(1), 77.

Walley AY, Xuan Z, Hackman HH, et al. Opioid overdose rates and implementation of overdose education and nasal naloxone distribution in Massachusetts: interrupted time series analysis. BMJ. 2013. Jan 30;346:f174.

White House. The Underestimated Cost of the Opioid Crisis. November 2017. https://www.whitehouse.gov/sites/whitehouse.gov/files/images/The%20Underestimated%20Cost%20of%20the%20Opioid%20Crisis.pdf