0930-0270 DTAC CCP Supporting Statement B (SSB)_06062025-DRAFT CLEAN

Crisis Counseling Assistance and Training Program Data Toolkit

SUPPORTING STATEMENT B

B. STATISTICAL METHODS

1. Sampling Methods

Ruggiero, K. J., Davidson, T. M., McCauley, J., Gros, K. S., Welsh, K., Price, M., Resnick, H. S., Danielson, C. K., Soltis, K., Galea, S., Kilpatrick, D. G., Saunders, B. E., Nissenboim, J., Muzzy, W., Fleeman, A., & Amstadter, A. B. (2015). Bounce Back Now! Protocol of a population-based randomized controlled trial to examine the efficacy of a web-based intervention with disaster-affected families. Contemporary Clinical Trials, 40, 138–149.

Efforts used to inform this section of the Office of Management and Budget (OMB) Supporting Statement include the following: (a) a retrospective evaluation of the Crisis Counseling Assistance and Training Program (CCP), (b) a cross-site evaluation of CCP grants funded as a result of the 2005 Gulf Coast hurricanes, and (c) analysis of CCP data from 2016 to 2024. The first two studies were performed by the National Center for Posttraumatic Stress Disorder (PTSD) as an interagency agreement with the Substance Abuse and Mental Health Services Administration (SAMHSA) Center for Mental Health Services (CMHS).

The retrospective evaluation involved an archival analysis of all available reports for CCP grants implemented in the 50 United States that closed out over a 5-year interval between October 1, 1996, and September 30, 2001. As such, all CCP grants that closed out during that interval and were administered by states were eligible for analysis. Reports from 44 grants covering 28 disaster events were included in the quantitative analysis for the retrospective evaluation.

Given the catastrophic nature of the 2005 Gulf Coast hurricanes, a disaster-specific cross-site evaluation of funded CCPs was supported by SAMHSA CMHS. The cross-site evaluation of the 2005 Gulf Coast hurricane programs consisted of 22 grants in 17 states active between November 2005 and February 2007, thus 3 to 18 months after Hurricane Katrina. These grants used the 2005 OMB-approved CCP data collection forms.

The means (or averages) from these studies, combined with a review of the data trends from the past 9 years and consideration of the current CCP grants, inform this section of the OMB Supporting Statement. The average number of responses from 2016 to 2024¹ was 227,200 per year for the CCPs, distributed as follows:

1. Individual/Family Crisis Counseling Services Encounter Log = 160,000

   1. Service provider staff members (i.e., crisis counselors/program staff) are required to complete this form for all participants who access individual or family crisis counseling services in sessions lasting 15 minutes or more.

2. Group Encounter Log = 13,200

   1. Service provider staff are required to complete this form for each group of participants who access group crisis counseling services and/or group public education services.

3. Weekly Tally Sheet = 41,600

   1. Service provider staff members are required to complete this form for all contacts for whom data are not captured on either the Individual/Family Crisis Counseling Services Encounter Log or Group Encounter Log.

4. Adult and Child/Youth Assessment and Referral Tools = 8,000 (7,200 for adults and 800 for children/youth)

   1. These tools (Adult or Child/Youth) can be administered at any time if the crisis counselor feels the participant is exhibiting distress or would benefit from referral to other services. It is recommended that the assessment and referral tools are administered during encounters where more than four event reactions or certain risk categories are indicated.

5. Participant Feedback Survey Form = 4,000

   1. These forms are collected from a sample of participants, not every participant. The use of a time-sampling approach is recommended, for example, soliciting participation from all counseling encounters during a 1-week period 6 months and 1 year post event.

6. Service Provider Feedback Form = 400

   1. These forms are administered to all service providers, and data are collected anonymously at least once at 6 months and/or 1 year after the disaster.

Most CCPs collect encounter information using the mobile app for fast, timely, and reliable data entry into the online system and expedited data analysis. The CCP forms can also be collected on paper and then entered directly into the online database or forwarded to a central location for online data entry. The completion of forms is a part of the daily work requirements for service provider staff and does not interfere with ongoing program operations.

2. Information Collection Procedures

CCP crisis counselors and outreach workers are responsible for completing the Individual/Family Crisis Counseling Services Encounter Log and the Group Encounter Log during the encounter or immediately after the participant(s) has/have completed the encounter session. Service provider staff members select appropriate response categories to items on the log based on their own perceptions of the participants or on information participants provide during the encounter. The Weekly Tally Sheet is completed by the CCP crisis counselor or outreach worker at the end of the designated “week” period (e.g., Sunday through Saturday) as determined by the CCP and the SAMHSA CMHS project officer.

The Individual/Family Crisis Counseling Services Encounter Log is completed by service provider staff for all participants who access individual or family crisis counseling services. The Group Encounter Log is completed by the service provider staff for all groups that meet for crisis counseling or for public education. The Weekly Tally Sheet is completed by the service provider staff for all brief educational or supportive encounters not captured by any other form.

The assessment and referral tools are administered by CCP service provider staff and used as a checklist with the participant present. The assessment and referral tools are intended to be completed by a trained service provider staff member for all participants who are exhibiting distress or would benefit from referral to other services. It is recommended that the assessment and referral tools be administered during encounters where more than four event reactions or certain risk categories are present. These tools are read aloud to the participant by the crisis counselor, and the participant is asked to respond to questions accordingly.

Both the Service Provider Feedback Form and Participant Feedback Survey Form are completed in a location of the respondent’s choosing (e.g., home, office) and anonymously. The Service Provider Feedback Form and Participant Feedback Survey Form are administered in English online. The Spanish versions are only administered in paper form, and English versions are available in paper form, as needed. No individual identifying information is collected on the forms to assure the anonymity of the respondent. The SAMHSA CMHS project officer along with Federal Emergency Management Agency staff provide guidance to the state in determining the most appropriate method for the collection, processing, and sharing of findings from these forms.

3. Methods to Maximize Response Rates

The following logs/tools are completed by a trained service provider staff member as part of their job requirements:

• Individual/Family Crisis Counseling Services Encounter Log

• Group Encounter Log

• Weekly Tally Sheet

• Adult and Child/Youth Assessment and Referral Tools

To maximize response rates and compliance with the completion of these tools, all crisis counselors are trained on the requirements of completing these forms as well as administration protocols. CCP grants put procedures in place for the oversight of the crisis counselors to ensure they complete all required forms. As such, our targeted response rate for the encounter logs (i.e., Individual/Family Crisis Counseling Services Encounter Log, Group Encounter Log, and Weekly Tally Sheet) and the Adult and Child/Youth Assessment and Referral Tools is 100 percent. However, a 100 percent completion rate may not always be possible for various reasons (e.g., the counselor may fail to complete a form as required, a form may not be available at the time of the encounter, a form may be lost). Therefore, we estimate a non-completion rate of between 2 and 5 percent, yielding a completion rate of approximately 95 percent for these forms.

The Participant Feedback Survey Form is completed by adult participants (for whom individual or group crisis counseling services were provided). The form is administered 6 and 12 months after the event. At those times, the program chooses a 1- or 2-week period during which selected sample members are asked to complete the Participant Feedback Survey Form anonymously. The response rate is calculated by comparing the total number of forms received and completed to the total number of forms that were distributed separately at 6 and 12 months. Methods to increase response rates include (a) providing a link and QR code for easy online access, and (b) providing an email from the program director encouraging participation. Furthermore, testing of the form has helped ensure that the form is easily comprehensible and brief enough to facilitate completion.

A review of previous data collected through the data forms indicates that the typical response rate for the Participant Feedback Survey Form is approximately 5 percent. Given this low response rate, it is likely that participants who complete the Participant Feedback Survey Form differ from participants who do not in ways important to the program. When findings are reported, the response rate and the likelihood of bias in the findings are communicated clearly, and aggregate-level characteristics of respondents as they relate to the program are provided.

The Service Provider Feedback Form is available via an online link to all CCP service providers at 6 months and/or 1 year after the disaster. Response rates are calculated separately for the 6-month and 1-year collection efforts. Methods to increase response rates among the service providers include an email to encourage completion of these forms.

4. Tests of Procedures

No new tests of procedures will be undertaken. Many instruments in the CCP Data Toolkit have been taken from established data collection tools that have already been tested for validity and reliability. In addition, SAMHSA CMHS staff members have had an opportunity to review the revised tools, and all agree with the data items. The Adult and Child/Youth Assessment and Referral Tools, the Participant Feedback Survey Form, and the Service Provider Feedback Form contain elements of the Short PTSD Rating Interview, or SPRINT, and the SPRINT-E, an expanded version of this tool, both of which have been determined in research to be reliable and internally consistent. The Child/Youth Assessment and Referral Tool has items from the University of California at Los Angeles PTSD Reaction Index (RI). The RI was updated in 2020 to align with the revised PTSD definition in the Diagnostic and Statistical Manual of Mental Disorders – fifth edition (DSM-5) and a very brief version of the RI-5 was developed. Thus, the Child/Youth Assessment and Referral Tool was updated to reflect the very brief RI-5 to ensure the most accurate screening and to reduce response burden. Other items on these forms, as well as on the Individual/Family Crisis Counseling Services Encounter Log and Group Encounter Log, evolved directly from previous studies (e.g., retrospective and cross-site evaluation of 2005 Gulf Coast hurricanes) and through site visits, interviews, and focus groups with states, direct service providers, and federal staff. Demographics collected across all forms are considered standard items for collection in the research literature and speak directly to the goals of the CCP.

5. Statistical Consultants

The names and phone numbers of project officers and the consultant are as follows:

Federal Project Officer

Anne Genet Reim, M.P.H.

Center for Mental Health Services

Division of Trauma and Disaster Behavioral Health

Disaster Behavioral Health Branch

5600 Fishers Lane

Rockville, MD 20857

Phone: 202–713–7866

anne.reim@samhsa.hhs.gov

Statistical Consultants

Abigail Woodroffe, Ph.D.

Principal Researcher

Manhattan Strategy Group

4747 Bethesda Avenue, Suite 1210

Bethesda, MD 20814

Phone: 202–403–5000

awoodroffe@manhattanstrategy.com

Alyssa Andrade, M.P.H.

Technical Assistance Consultant

Manhattan Strategy Group

4747 Bethesda Avenue, Suite 1210

Bethesda, MD 20814

Phone: 202–403–5000

aandrade@manhattanstrategy.com

1 Data from 2021 and 2022 were excluded from the analysis because of the COVID pandemic CCP grants. Because the pandemic affected all states, the number of grants and quantities of data collected were substantially higher than in other years.