B3 Attachment B3 - Grantee Trainee Survey -- NORC response

Grantee Trainee Survey OMB Number (0915-XXXX)

Expiration date (XX/XX/202X)

Note: The survey will start with a login page, followed by the Public Burden Statement, Introduction/ Consent, and Instructions. Then the survey will begin.

Public Burden Statement: This survey is intended to gather information from [GRANTEE PROGRAM NAME] trainees. The information gathered will contribute to the Bureau of Health Workforce (BHW) Substance Use Disorder (SUD) Evaluation. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this information collection is 0915-XXXX and it is valid until XX/XX/202X. This information collection is voluntary. Public reporting burden for this collection of information is estimated to average 20 minutes per response, including the time for reviewing instructions, searching existing data sources, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to HRSA Reports Clearance Officer, 5600 Fishers Lane, Room 14N136B, Rockville, Maryland, 20857 or paperwork@hrsa.gov.

Introduction/Consent:

Thank you for participating in our Bureau of Health Workforce Survey of Grantee Trainees. We value your input.

Your responses will be kept confidential. For all of the data we collect for analysis, we will use unique survey identifiers, not respondents’ names. Any published reports will summarize the results in the aggregate and will not include individual responses. At the end of the evaluation, all of the data that are collected will be provided to the Health Resources and Services Administration (HRSA).

Instructions:

Please use the “Continue” and “Previous” buttons to navigate through the questions in the survey. You must use the "Continue" button on the screen after you have responded to a question in order for your answer to be saved. Please do not use your browser buttons.

To exit the survey at any time, use the “Quit” button at the top of each screen. Using the “Quit” button saves your data and allows you to return to the same location later to complete the survey.

You may have trained at more than one site. Please provide responses about your experiences at the site where you spent most of your time. If you divided your time equally across sites, please provide responses based on your collective experience across sites.

[IF PARAPROFESSIONAL TRACK, DISPLAY: The following questions were developed for a wide range of programs. It’s possible that some of the questions may not seem like they apply to your experience in the [GRANTEE PROGRAM NAME]. If you come across a question that does not seem relevant to your experience, please mark the appropriate option, such as “not applicable,” and continue with the survey.]

We have provided definitions on certain terms throughout the survey. When available, you will see a question mark icon appear above a term where a definition is available. (OMB Reviewers: These definitions only appear at their first mention in this document to conserve space, but these definitions will be made available in the programmed web survey at every mention of the relevant term.)

Again, we greatly appreciate your time and participation. Let’s get started!

Preload variables required for survey administration:

Email Address

Grantee Program Name

Professional or Paraprofessional Track

1. Prior Experience and Motivation: We would like to know more about your experiences before applying to the [GRANTEE PROGRAM NAME].

1. Where did you consider home when you were ages 10 to 18? Please enter the country and, if the United States, enter the ZIP code or city/state.

[ ] Country __________________ [DROP DOWN OF COUNTRIES - PROGRAM UNITED STATES AT TOP; IF ANY COUNTRY OTHER THAN UNITED STATES IS SELECTED, SKIP TO Q2]

Enter ZIP code OR city/state

[ ] ZIP code _______________ [QC CHECK: LIMIT TO 5 DIGITS; IF ZIP CODE SELECTED, SKIP TO Q2]

[ ] City ______________/State______ [DROP DOWN OF STATES] [IF CITY/STATE SELECTED, SKIP TO Q2]

[ ] I moved around a lot

2. Did any of these factors influence your decision to apply to the [GRANTEE PROGRAM NAME]?

Please select a response for ALL rows.

Factors Influencing Application	Yes	No
Financial assistance (e.g., stipend, tuition)
Desire to work in a rural or underserved community
Desire to work in substance use disorder treatment and prevention
Mentorship opportunities with program faculty
Ability to receive experiential training in integrated care (HOVER OVER WEB FEATURE: Experiential training includes not only a designated faculty member’s instruction delivered to an individual or group of individuals, but also a component of direct work experience such as clinical practice-based experiences or supervised experiences in health care settings.) (HOVER OVER WEB FEATURE: Integrated care refers to a graduated spectrum of health services wherein a facility integrates primary care, behavioral health care, oral health care, and/or public health strategies into primary care settings.)
Prior work or training experience in a rural or underserved community
Experience living in a rural or underserved community
Career advancement
To prepare for a new career or job
Other: please specify_____________________
None of the above [DISALLOW IF OTHER OPTIONS SELECTED]		[DISALLOW]

3. [ASK IF MORE THAN ONE RESPONSE SELECTED AT Q2]
   Which of these factors was the PRIMARY factor influencing your decision?

[AUTOPOPULATE WITH SELECTIONS FROM Q2]
[ ] Don’t know

2. Classroom Instruction: This next set of questions asks about the classroom instruction provided as part of [GRANTEE PROGRAM NAME]. There will be a separate set of questions on your experiential training at training sites later in the survey, so please limit your responses to these questions to the instruction provided in the classroom (whether physical or virtual).

4. Which topics have been included in the classroom instruction provided in [GRANTEE PROGRAM NAME]?

Select ALL that apply.

[ ] Delivering integrated, interprofessional care

(HOVER OVER WEB FEATURE: Interprofessional training refers to two or more types of professionals learning about, from, and with each other to enable effective collaboration and improve health outcomes.)

[ ] Delivering team-based care

(HOVER OVER WEB FEATURE: Team-based care consists of intentionally created groups of at least three types of health providers with shared responsibility for a patient, group of patients, their families, and/or communities.)

[ ] Providing mental health dual diagnosis management

(WEB HOVER OVER FEATURE: Also known as "co-occurring disorder management")

[ ] Providing treatment services for substance use disorders, other than opioid use disorder

[ ] Providing opioid use disorder treatment services other than medication-assisted treatment

Note: Other terms for medication-assisted treatment (MAT) include medication for opioid use disorder (MOUD) and opioid agonist therapy (OAT). This survey uses MAT to cover all three terms.

[ ] Providing medication-assisted treatment (i.e., methadone, buprenorphine, and/or naltrexone) either individually or as part of a team

[ ] Developing care coordination plans for the integration of substance use disorder treatment and other medical and psychological needs

[ ] Providing care under different models of care delivery (e.g., varying levels of collaboration or integration among physical and behavioral health care providers)

[ ] Applying cultural and linguistic competency skills

(HOVER OVER WEB FEATURE: Cultural competency refers to knowledge, behaviors, attitudes, and policies that allow health professionals to understand/respect cultural differences and similarities such as by providing information in the language or cultural context most appropriate for the person being served.)

[ ] Applying strategies to reduce stigma around behavioral health issues

[ ] Delivering services through telehealth

(HOVER OVER WEB FEATURE: Teleheath refers to the use of electronic information and telecommunications such as videoconferencing or streaming media to support long-distance clinical health care and patient and professional health-related education.)

[ ] Pandemic emergency preparedness

[ ] Other: please specify _____________________

5. Do you feel prepared for each of the following because of the classroom instruction provided in [GRANTEE PROGRAM NAME]?

Please provide a response for ALL rows.

Classroom Topic	Yes, I feel prepared because of the classroom instruction	Yes, I feel prepared, but not because of the classroom instruction	No, I do not feel prepared
[POPULATE WITH TOPICS SELECTED IN Q4]

6. [ASK IF “YES, I FEEL PREPARED BECAUSE OF THE CLASSROOM INSTRUCTION” SELECTED FOR MORE THAN ONE RESPONSE IN Q5. POPULATE WITH ITEMS MARKED AS “YES, I FEEL PREPARED BECAUSE OF THE CLASSROOM INSTRUCTION” IN Q5]
   Which of these classroom topics has been MOST useful to you in preparing for your desired job?

[POPULATE WITH SELECTIONS FROM Q5]

[ ] Don’t know

7. Please indicate your level of agreement or disagreement with each of the following statements about the classroom instruction that is part of [GRANTEE PROGRAM NAME].

Please provide a response for ALL rows.

Classroom Instruction Component	Strongly agree	Agree	Disagree	Strongly disagree	Not applicable
I was satisfied with the quality of instruction
The instructor was available to answer questions
I was satisfied with the curriculum
My curriculum adequately prepared me to provide medication-assisted treatment either individually or as part of a team
My curriculum adequately prepared me for the type of health care position I’m seeking or in now

3. Experiential Training Experience: Now we would like to know about experiential training you received as part of [GRANTEE PROGRAM NAME].

(HOVER OVER WEB FEATURE: Experiential training includes not only a designated faculty member’s instruction delivered to an individual or group of individuals, but also a component of direct work experience such as clinical practice-based experiences or supervised experiences in health care settings.)

8. Do you (or did you) receive experiential training as part of [GRANTEE PROGRAM NAME]?

[ ] Yes

[ ] No [SKIP TO Q16]

9. At how many sites do you (or did you) receive experiential training? ______ [RANGE 1-25]

[ ] None

10. How many hours per week do you (or did you) spend in educational training at your training site(s) as part of [GRANTEE PROGRAM NAME]?

Number: ________ [RANGE 1-50]

[ ] None

[ ] Don’t know

11. [ASK IF TRACK=PROFESSIONAL] How many total clients or patients do you typically see per week at your experiential training site(s)?

[ ] None

[ ] 1 - 10

[ ] 11 – 20

[ ] 21 – 30

[ ] 31 – 40

[ ] 41 or more

12. [ASK IF TRACK=PROFESSIONAL] In which category (or categories) of service do you spend most of your time at the experiential training site(s)?

Please select ALL that apply.

[ ] Integrated, interprofessional behavioral health care delivery

[ ] Substance use disorder treatment services, excluding opioid use disorder treatment

[ ] Opioid use disorder treatment services, excluding medication-assisted treatment

[ ] Medication-assisted treatment for opioid use disorder

[ ] Administration of needs assessment for social services

[ ] Care coordination

[ ] Motivational interviewing

[ ] Telehealth

[ ] Administrative functions, including billing or scheduling

[ ] Other: please specify_____________________

[ ] None of the above

13. Which staff participate in interprofessional practice at your experiential training site?
    Please select ALL that apply.

[ ] Physician, excluding Psychiatrists (MD/Doctor of Medicine, DO/Doctor of Osteopathic Medicine)

[ ] Psychiatrist

[ ] Physician Assistant

[ ] Nurse Practitioner

[ ] Registered Nurse

[ ] Psychiatric Nurse Specialist

[ ] Certified Nurse-Midwife

[ ] Dentist (DDS/Doctor of Dental Surgery, DMD/Doctor of Medicine in Dentistry)

[ ] Dental Hygienist

[ ] Pharmacist

[ ] Clinical Psychologist

[ ] Substance Use Disorder Counselor

[ ] Marriage and Family Therapist

[ ] Licensed Clinical Social Worker

[ ] Other Licensed Professional Counselor

[ ] Behavioral Health Professional (not licensed)

[ ] Peer Provider

[ ] Administrative Staff

[ ] Other: please specify ___________________

[ ] There is no interprofessional practice at my site [DISALLOW IF ANOTHER OPTION SELECTED]

14. Have any of the following factors been challenges at your experiential training site?

Please provide a response for ALL rows.

Challenges	Yes, this is a challenge	No, this is not a challenge	Not applicable
The site administration lacks clarity of roles and responsibilities
There are rigid or inefficient management practices
There are not enough personnel at the site to meet patient loads
The site administration struggles with scheduling time for clinical training
There is a lack of staff training on opioid use disorder treatment
There is a lack of opioid use disorder services to meet clients’ full range of needs
There is a lack of capacity to meet the volume of clients’ opioid use disorder treatment needs
There is a lack of training opportunities in desired areas, aside from opioid use disorder treatment and services
There are limited opportunities to provide interprofessional, integrated care
There are limited opportunities to provide team-based care
There is a lack of workforce diversity (e.g., race, age)
The site’s location (e.g., distance, lack of transportation options, safety)
There is a lack of mentorship and/or supervision
Other: please specify_____________________		[DISALLOW]

15. [ASK IF “YES, THIS IS A CHALLENGE” SELECTED FOR MORE THAN ONE RESPONSE IN Q14]
    
    

Which factor has been the MOST CHALLENGING?

[AUTOPOPULATE WITH SELECTIONS FROM Q14]

[ ] Don’t know

4. COVID-19 Pandemic: This set of questions asks about your experiences during the COVID-19 pandemic.

16. How did your training change during the COVID-19 pandemic?

Please select ALL that apply.

[ ] My classroom instruction transitioned from in-person to online

[ ] I needed to complete the curriculum component of my training later than planned

[ ] I needed to complete my clinical training requirements later than planned

[ ] Other: please specify_____________________

[ ] My training did not change during the pandemic [DISALLOW IF ANOTHER OPTION SELECTED]

17. [ASK IF TRACK=PROFESSIONAL] During the COVID-19 pandemic, was there a shift in the types of services you provided at your experiential training site(s)?

Please provide a response for ALL rows.

Service	Provided more	Provided less	No change	Do not provide
Mental health visits, excluding conducting substance use disorder treatment
Substance use disorder treatment services, excluding opioid use disorder treatment
Opioid use disorder treatment services, excluding medication-assisted treatment
Medication-assisted treatment for opioid use disorder
Administration of needs assessment for social services
Care coordination
Motivational interviewing
Telehealth
Administrative functions, including billing or scheduling
Other: please specify___________________

[ASK IF ANY SERVICE WAS ‘PROVIDED MORE’ IN Q17] You indicated that you provided more of the following services during the COVID-19 pandemic. Did you provide these services before the COVID-19 pandemic?

18. Please provide a response for ALL rows.

Service	Yes, provided before COVID-19	No, did not provide before COVID-19
[POPULATE WITH SERVICES SELECTED AS ‘PROVIDED MORE’ IN Q17]

Beginning of Outcomes Section/Alumni Survey: These questions constitute the questions used to assess outcomes for trainees who have finished their training program and degree at the time of the trainee survey AND the entirety of the Alumni Survey. The Alumni Survey is administered to trainees who completed their training program but had not finished their degree when they completed the trainee survey.

5. Employment Outcomes and Job Placement: We have one more set of questions.

19. Have you completed your training for [GRANTEE PROGRAM NAME]?
    
    

[ ] Yes [IF TRACK PRELOAD = PARAPROFESSIONAL, SKIP TO Q25; IF TRACK PRELOAD = PROFESSIONAL, CONTINUE TO Q20]

[ ] No [SKIP TO Q38]

20. Have you finished your degree program?

[ ] Yes

[ ] No [SKIP TO Q39]

We would like to learn more about your professional experiences since completing the [GRANTEE PROGRAM NAME].

21. Are you currently employed?

[ ] Yes

[ ] No

22. [ASK IF Q21=NO] Are you actively seeking employment?

[ ] Yes

[ ] No

23. [ASK IF Q22=YES] Are you seeking employment in the same field in which you were trained through the [GRANTEE PROGRAM NAME]?

[ ] Yes

[ ] No

[ ] Don’t know

24. [Ask if Q22=NO] Please tell us the reason you are not currently seeking employment.

[ ] Have an offer in place, but haven’t started the job

[ ] Going back to school for ________________________ (Enter type of program)

[ ] Taking time off for personal reasons (e.g., family, travel)

[ ] Other: please specify___________________

For the rest of the section, we use the term “work” to mean providing the services for which you received training through the [GRANT PROGRAM NAME], whether it is paid or unpaid.

25. [Ask if Q21=YES OR TRACK PRELOAD=PARAPROFESSIONAL] Are you currently working in the same field in which you were trained through the [GRANTEE PROGRAM NAME]?

[ ] Yes

[ ] No

[ ] Don’t know

26. [ASK IF Q25=NO, DON’T KNOW] In which field are you currently working?

_______________________________________________________

27. [ASK if Q21=YES OR TRACK PRELOAD=PARAPROFESSIONAL] In what type of setting do you work? If you work in more than one setting, select the setting in which you spend most of your time.

[ ] Academic Institution

[ ] Critical Access Hospital

(HOVER OVER WEB FEATURE: A Critical Access Hospital (CAH), in general, must be located in a rural area; maintain no more than 25 inpatient beds and an average annual length of stay of 96 hours or less; furnish 24-hour emergency care services seven days a week; and be located either more than a 35-mile drive from the nearest hospital or CAH, or more than a 15-mile drive in areas with mountainous terrain or only secondary roads.)

[ ] Area Health Education Center

(HOVER OVER WEB FEATURE: Area Health Education Centers (AHECs) are public or nonprofit private organizations (e.g., hospitals, health organizations with accredited primary care training programs) that partner with an academic medical or nursing institution to recruit, train, and retain a health professions workforce committed to underserved populations. Most AHEC organizations include “AHEC” in their name.

[ ] Federally Qualified Health Center (FQHC) or “Look-Alike”

(HOVER OVER WEB FEATURE: FQHCs include 1) health care organizations that receive funding under the Public Health Services Act to provide primary health services and other related services to a population that is medically underserved; 2) FQHC “Look-Alikes,” which are nonprofit entities certified as meeting the requirements for receiving a grant under the Public Health Service Act, but are not grantees; and 3) outpatient health programs or facilities operated by a tribe, tribal organization, or by an urban Indian organization.) Many FQHCs have “Community Health Center” or “Health Center “as part of their name, and are.

[ ] Rural Health Clinic

[ ] Tribal Health Site

[ ] Other Clinical Training Site

[ ] Other: please specify___________________

28. [ASK IF Q25=YES AND TRACK=PROFESSIONAL] For how long have you worked there?

[ ] Less than 6 months

[ ] At least 6 months, but less than 1 year

[ ] 1 year or more

[ ] Don’t know

29. Since completing the [GRANTEE PROGRAM NAME], have you obtained a license to deliver relevant services?

[ ] Yes

[ ] No

30. [ASK IF Q29=NO] Please tell us the reason you have not obtained a license.

[ ] I am not required to have a license

[ ] I am still in school

[ ] I applied, but am waiting for approval

[ ] I applied, but was not approved

[ ] I had a license before completing the program

[ ] Other: please specify_____________________

31. [ASK IF TRACK=PROFESSIONAL] Do you have a Drug Addiction Treatment Act of 2000 waiver (DATA-2000 waiver)?

[ ] Yes

[ ] No

Now, a few more specific questions about your work experiences since completing the [GRANTEE PROGRAM NAME].

32. Since completing the [GRANTEE PROGRAM NAME], have you done any of the following?

Please provide a response for ALL rows.

Work/Service Experiences	Yes	No	Not applicable
A. Worked on a behavioral/mental health integrated care delivery team
B. Worked on an interprofessional care delivery team
C. Worked in a rural or medically underserved community

[ASK Q33-Q36 IF Q32C=YES]

33. Are you still working in a rural or underserved community?

[ ] Yes

[ ] No

34. [IF Q33=YES, FILL “HAVE YOU WORKED”; IF Q33=NO, FILL “DID YOU WORK”] Since completing your training, for how long [have you worked/did you work] in a rural or underserved community?

[ ] Less than 6 months

[ ] At least 6 months, but less than 1 year

[ ] 1 year or more

[ ] Don’t know

35. [IF Q33=YES, FILL “IS” AND “WORK”; IF Q33=NO, FILL “WAS” AND “WORKED”] What [is/was] the ZIP code of the place where you [work /worked]? If you [work/worked] in more than one place, answer about the place where you spent the most time. Enter the city/state if the ZIP code is not known.

Enter ZIP code OR city/state

[ ] ZIP code _______________ [QC CHECK: LIMIT TO 5 DIGITS; IF ZIP CODE SELECTED, SKIP TO Q36]

[ ] City ______________/State______ [DROP DOWN OF STATES] [IF CITY/STATE SELECTED, SKIP TO Q36]

36. [ASK IF TRACK=PROFESSIONAL] To what extent did each of the following factors influence your decision to work in a rural or underserved community after completing your program?
    Please provide a response for ALL rows.

Influences on Work/Service Decisions	A lot	Some	A little	Not at all	Don’t know
Working with a rural or underserved population
Living closer to family or friends
Working on an interprofessional, integrated care delivery team
Working in a setting with an organizational culture or management style that I like
The opportunity to apply a diverse skill set
Working at a site that offers financial incentives (e.g., competitive wages, retirement benefits)
Living in a community with features that are important to me personally (e.g., affordable housing, transportation, recreation)
Working in a site that offers professional benefits (e.g., continuing education)
Working in a site that offers a flexible schedule with work/life balance

37. In which of these roles, if any, do you see yourself one year from now?

Please select ALL that apply.

[ ] Providing behavioral health care, excluding substance use disorder services

[ ] Providing substance use disorder services, excluding opioid use disorder services

[ ] Providing opioid use disorder services

[ ] Enrolled in training that offers a more advanced educational certificate/degree

[ ] In a faculty or teaching position in behavioral health

[ ] In some other discipline (i.e., not in behavioral health)

[ ] Other: please specify___________________

[ ] I don’t know which role I see myself in [DISALLOW IF OTHER OPTIONS SELECTED]

38. [ASK IF Q19=NO] When do you anticipate finishing [GRANTEE PROGRAM NAME]?

_________Month [DROP DOWN OF MONTHS]

_________Year [RANGE 2020-2030]

[ ] Don’t know

39. [ASK IF Q20=NO] When do you anticipate finishing your current degree program?

_________Month [DROP DOWN OF MONTHS]

_________Year [RANGE 2020-2030]

[ ] Don’t know

40. [ASK IF Q19 or Q20=NO] We very much would like to follow up with you with a brief survey when you complete your degree program to hear more about your professional experiences. Please provide the best two email addresses to reach you for a follow-up survey:

____________________Primary email

____________________Secondary email

Please click on the “Submit” button to submit your responses. Once submitted, your answers cannot be changed.

[END] That was the last question; thank you again for participating in our survey!

18