Instrument 17: Follow-up protocol for child assessments (child burden)

VIQI Follow-up Protocol for Child Assessments

April 2021

FOLLOW-UP PROTOCOL FOR CHILD ASSESSMENTS

The purpose of the follow-up child assessments is to collect data on the sample children’s skills at the end (follow-up) of the impact evaluation and process study. These data will allow us to measure gains across the school year in children’s skills, to assess the impacts the different interventions have on child skills, and to explore the nature of the quality-to-child outcomes relationship. We will do this by comparing child assessment data from the program group to the control group and conducting secondary analyses with the data.

Direct Child Assessments

Overview: During the impact evaluation and process study, assessors will visit all classrooms participating in the VIQI Project to conduct direct child assessments. Follow-up assessments will take place in spring and will last up to an hour per child. Prior to beginning assessments, field staff will follow the protocol outlined below to schedule assessments with school staff.

SCHEDULING PROTOCOL

Scheduling with School Staff: Field staff will contact Program Administrators, Center Administrators, and teachers to schedule assessments. They will communicate with participants via letters, emails, and phone calls.

Program Administrator Communications: Field staff will call or send a letter to Program Administrators to give them information about the direct child assessments. The following information will be included in communications with Program Administrators to schedule assessments:

• Notification about the upcoming child assessments

• A reminder about the VIQI Study, stressing the study background, importance of the study, and the study sponsors

• A detailed description of all components of the child assessment visit

• The data collection schedule

• The VIQI Study website

• The toll free study number and study email account for questions

Center Administrator Communications: Field staff will email Center Administrators to give them information about the assessment data collection procedures. This email will include details about the child assessment visit and the timeline for assessments.

Teacher Communications: Field staff will email teachers to give them information about the assessment data collection procedures. This email will include details about the child assessment visit and the timeline for assessments and will request potential target weeks for when assessments can be conducted.

Parent Communications: Prior to data collection beginning, field staff will send home a reminder to parents via backpack mail reminding parents that they gave us permission to conduct the child assessments and that we are planning to conduct those assessments again in the spring. See below for a draft of this reminder letter.

REMINDER LETTER TO PARENTS FOR FOLLOW-UP ASSESSMENTS

Dear Parent/Guardian,

Thank you so much for allowing your child to participate in the VIQI Study (Variations in Implementation of Quality Interventions)! Your help is so valuable to this important study. You might remember that last fall/winter you gave us permission to play some games with your child, like naming pictures, counting objects, and sorting cards.

We are planning to play similar games with your child again this spring. The games will take place during the day at your child’s early care and education center and may take up to 60 minutes. Your child will get stickers to thank him/her for playing the games.

Remember that participation is completely voluntary. Please let us know if you or your child would like to stop participating.

As always, if you have any questions about your child’s participation in this project, please call or email [ADD STUDY CONTACT INFO]. This study is being conducted by a team of staff from MDRC, Abt Associates, RTI International, and MEF Associates.

Thank you again!

Sincerely,

The VIQI Study Team

[Note: a Spanish translation of the letter will be provided to families that report speaking Spanish at home and/or whose children were assessed in Spanish at baseline.]

ASSESSMENT PROTOCOL

Assessment Procedures: On the day of direct child assessments, observers will meet with teachers to explain the assessment procedures, using the talking points listed below. Assessors will ask teachers to introduce them to the children being assessed in the classroom. Assessors will make small talk with children in the classroom, beginning to build rapport, before bringing them to the assessment room (a predetermined spot in the classroom where assessments can be conducted without interruption or distractions).Throughout the assessment, assessors will offer children stickers and breaks as the child needs them.

Teacher Talking Points:

• Overview: We will conduct child assessments with selected children in your classroom who have consented to participate in the VIQI Study. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0508 and the expiration date is XX/XX/XXXX. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to [Contractor Contact Name]; [Contractor Contact Address].

• Assessment Purpose: These assessments will be used to help us build evidence about how to best improve the quality of early child care and education.

• Assessment Experience: The assessments are structured as games, similar to ‘Simon Says.’ Children will be asked to do things like sort pictures and count objects.

• Assessment Length: The assessments will take an hour, at most, for each child in the study.

• Privacy: The research team will do everything possible to keep the identity of the children and classroom private, to the extent permitted by law. Only the research team will view the assessment data. Child and classroom names will never be named in reports or other publications or presentations.

• Voluntary: The child assessments are voluntary. The child and his or her parents choose whether to participate or not, and may opt out at any point in the study. By being on my list for assessments, this indicates that the parent or guardian already gave permission for this child to participate in assessments.

Example Child Assessment Introduction Script: Before administering the child assessment, assessors will briefly explain what the activities will entail. The script assessors will use is provided below. This explanation will not only tell the child what the assessor will be doing and that some of the activities may be difficult, but it will also be a way of obtaining the child’s assent. The assessor will use the following script at the start of assessments to introduce the process to children:

“Hello, [CHILD NAME], my name is [ASSESSOR NAME]. Today, we will be playing some games. I will have some pictures to show you and some things I will ask you to do. Please listen carefully and do the best you can. Some of the things I will ask you to do are hard, even for older kids. So, don’t worry if you’re not sure about them. Just give it your best try. I have some stickers here for you when we are finished with the different parts. Are you ready to begin?”

If after the language screener, the child is routed into the Spanish versions of the battery, the assessor will introduce the process to children using the script above in Spanish.

Reluctant Respondents: We expect most children to be friendly and willing to cooperate, but it is expected that some may be hesitant or fearful of the process. Assessors will be trained to become skilled at identifying children’s reluctance before it escalates. When put into an assessment situation, some children may become stressed or anxious. Others may leave the table to run around. By being attentive to the child who is slow to warm up or easily distracted, assessors can potentially avoid a situation in which they are unwilling to continue. Assessors will learn to recognize when a child may need a break and that by stopping for a few minutes, the child may become re-engaged in the assessment and not end up refusing. Some children may just be having a bad day, so returning at another time may be an appropriate option.

Example Child Assessment End Script: Assessors will end the assessment by thanking children and giving them stickers. The assessor will use the following script at the end of assessments to thank children for their hard work:

“Congrats – we’re all finished! You did a great job. Thank you so much for your hard work! I have some stickers here for you. Please pick a few, and then I will bring you back to your classroom.”

FOLLOW-UP ASSESSMENT BATTERY

The team will administer assessments at follow-up assessing the following domains. Assessments will be finalized depending on whether assessments need to be conducted remotely due to COVID-19The combination of assessments chosen for the follow-up assessment battery will take no more than 60 minutes. Assessments might include:

• Language/Literacy (we will select 1measure)

  • Preschool Language Scales, 5^th Edition (PLS) and the PLS Spanish (Zimmerman, Steiner, & Pond, 2011; Zimmerman, Steiner, & Pond, 2012) assess language development in children ages birth to 7:11 with Auditory Comprehension and Expressive Communication subscales.

  • Quick Interactive Language Screener (QUILS) and the QUILS: English-Spanish (Golinkoff, de Villiers, Hirsh-Pasek, Iglesias & Wilson, 2017; Iglesias, de Villiers, Golinkoff, Hirsh-Pasek & Wilson, in press) assess language development in children ages 3 to 5 with Vocabulary, Syntax, and Process subscales.

• Mathematics

  • Child Math Assessment (CMA) (Klein & Starkey, 2004; Starkey & Klein, 2012) assesses number, operations, shape and patterns.

• Science (may select if there is time in the battery)

  • CIRCLE Progress Monitoring/C-PALLS+ science screener for children aged 3 and 4

• Executive Function (we will select up to 2).

  • Pencil Tap (Diamond & Taylor, 1996) is a measure of executive functioning that captures inhibitory control, working memory, and attention.

  • Minnesota Executive Function Scale (MEFS; Carlson & Zelazo, 2014) is a measure of executive function that measures cognitive flexibility, working memory, and inhibitory control skills.

• Social-Emotional

  • Leiter-R social-emotional rating subscale examiner version (Roid & Miller, 1997) is an assessor report that assesses children’s behavioral/emotional regulation and sustained attention. It is completed by assessors about the child after the child finishes the general battery of assessments.

• We will also begin the battery by collecting either the Simon Says and Art Show subscales of the preLAS or the QUILS language screener as a language screener to determine whether the child receives the rest of the battery in English or Spanish.

4