NonSub Change - SAS-HV Feedback Updates

To: Kelsi Feltz

Office of Information and Regulatory Affairs (OIRA)

Office of Management and Budget (OMB)

From: Shirley Adelstein

Office of Planning, Research & Evaluation (OPRE)

Administration for Children and Families (ACF)

Date: March 28, 2024

Subject: Change Request – Supporting and Strengthening the Home Visiting Workforce (SAS-HV): Testing and Validation of a Draft Measure of Reflective Supervision for Home Visiting (OMB #0970-0355)

This memo requests approval of changes to the approved information collection, Supporting and Strengthening the Home Visiting Workforce (SAS-HV): Testing and Validation of a Draft Measure of Reflective Supervision for Home Visiting, approved under the Pre-testing of Evaluation Data Collection Activities (OMB #0970-0355).

Background

Under this umbrella generic OMB has approved multiple efforts related to pretesting and validating a draft measure of reflective supervision for Home Visiting. This includes an initial request approved August 19, 2022 to complete early concept mapping. A second request for piloting a measure was approved June 12, 2023. And most recently (January 30, 2024), OMB approved efforts to test and validate a draft measure.

Forty-one home visiting supervisors pilot tested the SAS-HV reflective supervision measure and provided feedback on their experience completing the draft measure, as approved by OMB on June 12, 2023. Based on the supervisors’ feedback, preliminary analyses of the data, and internal planning efforts, we identified some changes to the current instruments for the most recently approved information collection effort.

Overview of Requested Changes

Based on the feedback, the team identified four repetitive items, four instances of unclear language, two items that would fit better in a different section, and a set of response options that were unclear. From these findings, we also identified the need to ask additional demographic and background questions that would better allow us to explore patterns in the results (Instrument 1). Two supervisors reviewed the measures to examine convergent and concurrent validity (Instrument 5), and recommended expanding “clients” to “families” in one measure and switching the order of two measures. During internal planning meetings, we identified a need to add additional questions to help recruit home visitors and track responses over time for the repeated measures aspect of this study (Instrument 6).

Changes to Instrument 1: Participant Questionnaire

• Added three questions:

  • How often do you typically provide individual supervision to home visitors that you supervise? (Question #5)

    • Once a week

    • Every other week

    • Once a month

    • Every other month

    • Other, please specify:

  • Do you work as a supervisor for your home visiting program or as a consultant? (Question #18)

    • Supervisor

    • Consultant

  • What training have you participated in (check all that apply)? (Question #2 under the heading “Lastly, we would like to...”)

    • Reflective Interaction Observation Scale (RIOS)

    • Infant Mental Health endorsement as an infant-family reflective supervisor

    • Facilitating Attuned Interactions (FAN)

    • Model-specific reflective supervision training

    • Other, please describe: text box

Changes to Instrument 3: RS Measure

• Moved two items to a different location in the measure:

  • Question #9 was previously #42

  • Question #10 was previously #42

• Revised question stem for Items 11 through 15

• Removed items

  • How often, if at all, did you discuss home visitor’s power and privilege?

  • How often, if at all, did you discuss home visitor’s culture?

  • How often, if at all, did you model techniques to use when working with families?

  • How often, if at all, did you provide examples of potential strategies for working with a family?

• Changed response options for four items

  • Changed response options from “yes/no” to “yes/not in this session” in Questions #16-19

• Changed language for four items

  • Revised wording from “home visitor’s racial and ethnic identity” to “home visitor’s personal identities (e.g., race, ethnicity, culture, or gender)” in Question #13

  • Revised wording from “home visitor’s implicit biases” to “home visitor’s assumptions or beliefs about a family based on family characteristics (e.g., race, ethnicity, culture, gender)” in Question #14

  • Replaced the word “technique” with the word “strategy” in Question #25

  • Replaced the word “their” with the phrase “a family’s” in Questions #48 and 49

Changes to Instrument 4: Participant Perspectives

• Added four questions:

  • About how long have you been providing reflective supervision to this home visitor?  

    • Less than 3 months 

    • Between 3 months and 6 months 

    • More than 6 months but less than one year 

    • One year or more 

  • Did you serve as a home visitor alongside this home visitor prior to becoming their supervisor?

    • Yes

    • No 

  • Does the home visitor identify as Hispanic, Latino, or Spanish origin?  

    • Yes

    • No

    • Not sure, the home visitor has not shared this information 

    • Prefer not to answer 

  • Which race does the home visitor identify as? 

    • American Indian Alaska Native 

    • Asian 

    • Black or African American 

    • Native Hawaiian Other Pacific Islander 

    • White 

    • Other, please describe 

    • Not sure, the home visitor has not shared this information 

    • Prefer not to answer 

Changes to Instrument 5: Measures to Examine Convergent and Concurrent Validity

• Changed language slightly for seven items

  • Added “or family[ies]” after the word “client[s]” in Questions #1, 7, 8, 9, 10, 12, and 22

• Re-ordered two measures

  • Moved the Supervisory Working Alliance Inventory: Supervisor Form in front of the Reflective Supervision Self-Assessment Scale

Changes to Instrument 6: Questionnaire for Repeated Measures

• Added three questions:

  • Please provide the home visitor’s email address:

  • Which session are you reporting on for this home visitor?

    • First

    • Second

    • Third

  • Did you serve as a home visitor alongside this home visitor prior to becoming their supervisor? 

    • Yes

    • No

Time Sensitivities

An informational webinar is scheduled for Tuesday, April 2, in preparation for the survey launch on Monday, April 8. We would like to complete final internal review and testing of data collection instruments by Friday April 5^th and kindly request OMB approval prior to Friday April 5^th.