NonSub Change - SAS-HV: Testing and Validation Instrument Updates

To: Kelsi Feltz

Office of Information and Regulatory Affairs (OIRA)

Office of Management and Budget (OMB)

From: Shirley Adelstein

Office of Planning, Research & Evaluation (OPRE)

Administration for Children and Families (ACF)

Date: March 1, 2024

Subject: Change Request – Supporting and Strengthening the Home Visiting Workforce (SAS-HV): Testing and Validation of a Draft Measure of Reflective Supervision for Home Visiting (OMB #0970-0355)

This memo requests approval of changes to the approved information collection, Supporting and Strengthening the Home Visiting Workforce (SAS-HV): Testing and Validation of a Draft Measure of Reflective Supervision for Home Visiting, approved under the Pre-testing of Evaluation Data Collection Activities (OMB #0970-0355).

Background

Under this umbrella generic OMB has approved multiple efforts related to pretesting and validating a draft measure of reflective supervision for Home Visiting. This includes an initial request approved August 19, 2022, to complete early concept mapping. A second request for piloting a measure was approved June 12, 2023. And most recently (January 30, 2024), OMB approved efforts to test and validate a draft measure.

Forty-one home visiting supervisors pilot tested the SAS-HV reflective supervision measure and provided feedback on their experience completing the draft measure, as approved by OMB on June 12, 2023. Based on the supervisors’ feedback, preliminary analyses of the data, and internal planning efforts, we identified some changes to the current instruments for the most recently approved information collection effort.

Overview of Requested Changes

Based on feedback, the team identified two repetitive items, three instances of unclear language, two items that would fit better in a different section, and a set of response options that were unclear. From these findings, we also identified the need to ask additional demographic and background questions that would better allow us to explore patterns in the results (Instrument 1). Two supervisors reviewed the measures to examine convergent and concurrent validity (Instrument 5), and recommended expanding “clients” to “families” in one measure and switching the order of two measures. During internal planning meetings, we identified a need to add a question to help recruit home visitors for the repeated measures aspect of this study (Instrument 6).

Changes to Instrument 1: Participant Questionnaire

• Added two questions:

  • Do you work as a supervisor for your home visiting program or as a consultant? (Question #17)

    • Supervisor

    • Consultant

  • What training have you participated in (check all that apply)? (Question #2 under the heading “Lastly, we would like to...”)

    • Reflective Interaction Observation Scale (RIOS)

    • Infant Mental Health endorsement as an infant-family reflective supervisor

    • Facilitating Attuned Interactions (FAN)

    • Model-specific reflective supervision training

    • Other, please describe: [text box]

Changes to Instrument 3: RS Measure

• Moved two items to a different location in the measure:

  • Question #9 was previously #42

  • Question #10 was previously #42

• Removed items

  • How often, if at all did you model techniques to use when working with families?

  • How often, if at all, did you provide examples of potential strategies for working with a family?

• Change response options for four items

  • Changed response options from “yes/no” to “yes/not in this session” in Questions #16-19

• Changed language slightly for three items

  • Replaced the word “technique” with the word “strategy” in Question #25

  • Replaced the word “their” with the phrase “a family’s” in Questions #48 and 49

Changes to Instrument 4: Participant Perspectives

• Added one question:

  • About how long have you been providing reflective supervision to this home visitor?  

    • Less than 3 months 

    • Between 3 months and 6 months 

    • More than 6 months but less than one year 

    • One year or more 

Changes to Instrument 5: Measures to Examine Convergent and Concurrent Validity

• Changed language slightly for seven items

  • Added “or family[ies]” after the word “client[s]” in Questions #1, 7, 8, 9, 10, 12, and 22

• Re-ordered two measures

  • Moved the Supervisory Working Alliance Inventory: Supervisor Form in front of the Reflective Supervision Self-Assessment Scale

Changes to Instrument 6: Questionnaire for Repeated Measures

• Added one question:

  • Please provide the email address of the supervisee: [text box]

Time Sensitivities

An informational webinar is scheduled for Tuesday, April 2, in preparation for the survey launch on Monday, April 8. We would like to complete final internal review and testing of data collection instruments by Friday, March 29 and kindly request OMB approval prior to Friday, March 29.