DTAC Event OMB SS-B 1-8-2014

SAMHSA Disaster Technical Assistance Center (DTAC)

Training, Webinar, Podcast, and Mobile Application Feedback Forms

Supporting Statement

B. Statistical Methods

B1. Respondent Universe and Sampling Methods

Training Feedback Form. Participants in all SAMHSA DTAC web-based training, as well as all in-person training, will be asked to participate in the Training Feedback Form. For web-based trainings, participants’ email addresses may be collected upon registering for the training. If the form is being administered by email, this contact information will be used to send an invitation with a link to the web-based form. For forms administered through the final page of the web-based training and for forms completed via paper-and-pencil, participants’ phone numbers and email addresses will be collected upon registration for or attendance at the event. This contact information will be stored in a limited-access database and used to administer the Training Feedback Follow-Up Interviews.

Webinar/Podcast Feedback Form. Participants and viewers of all SAMHSA DTAC webinars and podcasts will be asked to participate in the Webinar/Podcast Feedback Form. For webinars, participants’ email addresses may be collected upon registering for the webinar. If the form is being administered by email, this contact information will be used to send an invitation with a link to the web-based form. For forms administered through the final page of the webinar, participants’ phone numbers and email addresses will be collected upon registration for the event. This contact information will be stored in a limited-access database and used to administer the Webinar Feedback Follow-Up Interviews.

Mobile Application Feedback Form. All individuals who download and use the SAMHSA mobile application will be asked to participate in the Mobile Application Feedback Form.

Training Feedback Follow-Up Interviews. A sample of up to 10 percent of the participants from each training will be selected for participation in the Training Feedback Follow-Up Interviews (or a minimum of 5 participants if 10 percent equals fewer than 5 participants). Individuals who participated in a Training Feedback Follow-Up Interview within the previous year will not be solicited for participation in the subsequent administration. Individuals will be contacted by email or telephone and asked to participate in the interview.

Webinar Feedback Follow-Up Interviews. A sample of up to 10 percent of the participants from each webinar will be selected for participation in the Webinar Feedback Follow-Up Interviews (or a minimum of 5 participants if 10 percent equals fewer than 5 participants). Individuals who participated in a Webinar Feedback Follow-Up Interview within the previous year will not be solicited for participation in the subsequent administration. Individuals will be contacted by email or telephone and asked to participate in the interview.

B2. Procedures for Collection of Information

The SAMHSA DTAC form team will be responsible for managing all data collection instruments.

Training Feedback Form. Web-based training participants will receive an invitation to participate in the form either immediately following the event (within the online training window), or via email shortly after the event (within one business day). The Training Feedback Form will be administered to all participants (i.e., a census). It is anticipated that form participants will complete the forms during work hours and while sitting in their workspaces. The first page of the form will contain the consent to participate form that participants will be required to read and agree to before participating. This form will contain language about voluntary participation. Only participants who agree to participate by checking the “Start Survey” box on the form landing page will be directed to the form. Followup reminder emails may be sent to study participants who have not yet completed the forms, as needed during the data collection period.

For in-person training participants, the form will be administered immediately following the training event by the trainer or a member of the SAMHSA DTAC team at the training. Participants will be provided a hard copy of the form instrument to complete and return to the trainer or SAMHSA DTAC team member. Forms will contain a cover page for providing informed consent. This cover page will contain language about voluntary participation. Only participants who sign the informed consent document will be asked to participate in the form.

Webinar/Podcast Feedback Form. Participants will receive an invitation to participate in the form either immediately following the event (within the webinar or podcast window), or via email shortly after the event (within one business day). The forms will be administered to all participants (i.e., a census). It is anticipated that form participants will complete the forms during work hours and while sitting in their workspaces. The first page of the form will contain the consent to participate form that participants will be required to read and agree to before participating. This form will contain language about voluntary participation. Only participants who agree to participate by checking the “Start Survey” box on the form landing page will be directed to the form. Followup reminder emails may be sent to study participants who have not yet completed the forms, as needed during the data collection period.

Mobile Application Feedback Form. Users of the mobile application will receive a pop-up form directly through the application. The form will periodically be displayed to users as they are accessing the application. The first page of the pop-up form will contain language about voluntary participation. From this page, users will have the ability to complete the form, select to have the form displayed again at a later time, or select to not participate in the form. Only those who select to complete the form immediately will be directed to the form. Followup reminders may be sent to users periodically as they access the application.

Training Feedback Follow-Up Interviews. Up to 10 percent of the training participants (or a minimum of 5 participants if 10 percent equals fewer than 5 participants) will be asked to participate in the Training Feedback Follow-Up Interviews. Prior to selecting the sample, individuals who participated in a Training Feedback Follow-Up Interview within the past year will be removed from the sample frame. Participants will then be randomly sampled from the remaining sample frame. The SAMHSA DTAC form team will contact sample members via phone or email to invite them to participate in the interview. The SAMHSA DTAC form team will provide potential participants with information on the purpose of the interview and, if the invitation is sent via email, it will include a telephone number and email address to contact the SAMHSA DTAC form team to schedule an interview. It is anticipated that indepth interview participants will complete the interview during work hours and while sitting in their workspaces. Followup reminder phone calls and emails will be sent to study participants who have not yet contacted the SAMHSA DTAC form team to schedule an interview, as needed during the data collection period.

Training Feedback Follow-Up Interviews and Webinar Feedback Follow-Up Interviews. Up to 10 percent of the training participants (or a minimum of 5 participants if 10 percent equals fewer than 5 participants) will be asked to participate in the Training Feedback Follow-Up Interviews, and up to 10 percent of the webinar participants (or a minimum of 5 participants if 10 percent equals fewer than 5 participants) will be asked to participate in the Webinar Feedback Follow-Up Interviews. Prior to selecting the sample, individuals who participated in a Qualitative Follow-Up Interview within the past year will be removed from the sample frame. Participants will then be randomly sampled from the remaining sample frame. The SAMHSA DTAC form team will contact sample members via phone or email to invite them to participate in the form. The SAMHSA DTAC form team will provide potential participants with information on the purpose of the interview and, if the invitation is sent via email, it will include a telephone number and email address to contact the SAMHSA DTAC form team to schedule an interview. It is anticipated that indepth interview participants will complete the interview during work hours and while sitting in their workspaces. Followup reminder phone calls and emails will be sent to study participants who have not yet contacted the SAMHSA DTAC form team to schedule an interview, as needed during the data collection period.

B3. Methods to Maximize Response

Training Feedback Form and Webinar/Podcast Feedback Form. Several steps will be taken to increase the form response rate. For those forms administered by email, these include email notifications, a customized introductory email with the form link, functionality to start and stop the form without losing responses, and a form helpdesk to assist with technical questions. For those administered through the TWP window, these include functionality to start and stop the form without losing responses and a form helpdesk to assist with technical questions. To maximize accessibility, the form will remain open for 1 month. Reminder emails will be used to increase participation rates. A paper version of the form will be available to anyone who requests it. Throughout the period that the form is open, a form helpdesk email address (DTACsurvey@icfi.com) will be available to participants to address any questions they may have as they are completing the form.

To maximize participation for the in-person training feedback forms, the forms will be administered to participants at the completion of the training. Clear instructions for completing the form and information on voluntary participation will be provided verbally to all training participants prior to passing out the forms.

Training Feedback Follow-Up Interviews, Webinar Feedback Follow-Up Interviews, and Qualitative Follow-Up Interviews. To maximize participation, the interviews will be scheduled over a one-month period. Confirmation and reminder emails will be sent to potential respondents to increase participation rates. Interviews will be scheduled Monday through Friday, from 8:00 a.m. to 5:30 p.m. ET to accommodate the business hours of most potential participants. Requests for other interview times will be handled on a case-by-case basis. Potential participants will be contacted by telephone and email and asked to participate in a 20–30 minute interview. Those contacted via email will be provided a telephone number and an email address to use in scheduling the interviews. Multiple contact attempts will be made. An individual will be available to respond to phone and email requests from 8:00 a.m. to 5:30 p.m. ET, Monday through Friday, and a voicemail will be set up to accommodate calls during non-business hours.

B4. Tests of Procedures

The ICF Survey Research Center conducted pilot testing of the data collection instruments. Pilot testing and revisions took place prior to OMB and institutional review board (IRB) submission. All instruments were piloted with participants similar to the intended respondents and in a similar office environment. Pilot testing of the instruments was based on evaluating field-related online trainings in which all pilot participants had recently participated. Following pilot testing, the data collection tools were refined to make them as clear as possible. In addition, all form data collection instruments have been reviewed by form research methodologists and technical assistance (TA) specialists for content and methodological review. All components were tested to determine average burden estimates.

B5. Statistical Consultants

SAMHSA DTAC has full responsibility for the development of the overall statistical design and assumes oversight responsibility for data collection and analysis. The individuals responsible for overseeing instrument design and data collection are the following:

Design/Collection/Analysis

Amy E. Falcone

Senior Associate

ICF International

9300 Lee Highway

Fairfax, VA 22031

Tel. 703-934-3935

Michael A. Lodato

Manager

ICF International

9300 Lee Highway

Fairfax, VA 22031

Tel. 703-934-3794

Project and Task Management

Amy R. Mack

SAMHSA DTAC Project Director

ICF International

9300 Lee Highway

Fairfax, VA 22031

Tel. 202-294-1431

The SAMHSA Project Officers responsible for receiving and approving deliverables are Nikki Bellamy and Erik Hierholzer:

Nikki Bellamy, Ph.D.

Center for Mental Health Services

SAMHSA

1 Choke Cherry Road

Room 6-1107

Rockville, MD 20857

Tel. 240-276-2418

Erik Hierholzer

Project Management Officer

SAMHSA

1 Choke Cherry Rd, 6-1002

Rockville, MD 20857

Tel. 240-276-1880

List of Attachments

Attachment A1 – Training Feedback Form

Attachment A2 – Webinar/Podcast Feedback Form

Attachment A3 – Mobile Application Feedback Form

Attachment B1 – Training Feedback Follow-Up Interview Guide

Attachment B2 – Webinar Feedback Follow-Up Interview Guide

____________________________________________________________________________

Page 5 of 5