Attachment C.5: Follow-up protocol for child assessments (child burden)

Attachment C.5: VIQI Follow-up Protocol for Child Assessments

April 2018

Attachment C.5

FOLLOW-UP PROTOCOL FOR CHILD ASSESSMENTS

The purpose of the follow-up child assessments is to collect data on the sample children’s skills at the end (follow-up) of the pilot study and the impact evaluation and process study. These data will allow us to measure gains across the school year in children’s skills, to assess the impacts the different interventions have on child skills, and to explore the nature of the quality-to-child outcomes relationship. We will do this by comparing child assessment data from the program group to the control group and conducting secondary analyses with the data.

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-XXXX and the expiration date is XX/XX/XXXX.

Direct Child Assessments

Overview: During the pilot study and impact evaluation and process study, assessors will visit all classrooms participating in the VIQI Project to conduct direct child assessments. Follow-up assessments will take place in spring and will last up to an hour per child. Prior to beginning assessments, field staff will follow the protocol outlined below to schedule assessments with school staff.

SCHEDULING PROTOCOL

Scheduling with School Staff: Field staff will contact Program Administrators, Center Administrators, and teachers to schedule assessments. They will communicate with participants via letters, emails, and phone calls.

Program Administrator Communications: Field staff will call or send a letter to Program Administrators to give them information about the direct child assessments. The following information will be included in communications with Program Administrators to schedule assessments:

• Notification about the upcoming child assessments

• A reminder about the VIQI Study, stressing the study background, importance of the study, and the study sponsors

• A detailed description of all components of the child assessment visit

• The data collection schedule

• The VIQI Study website

• The toll free study number and study email account for questions

Center Administrator Communications: Field staff will email Center Administrators to give them information about the assessment data collection procedures. This email will include details about the child assessment visit and the timeline for assessments.

Teacher Communications: Field staff will email teachers to give them information about the assessment data collection procedures. This email will include details about the child assessment visit and the timeline for assessments and will request potential target weeks for when assessments can be conducted.

Parent Communications: Prior to data collection beginning, field staff will send home a reminder to parents via backpack mail reminding parents that they gave us permission to conduct the child assessments and that we are planning to conduct those assessments again in the spring. See below for a draft of this reminder letter.

REMINDER LETTER TO PARENTS FOR FOLLOW-UP ASSESSMENTS

Dear Parent/Guardian,

Thank you so much for allowing your child to participate in the VIQI Study (Variations in Implementation of Quality Interventions)! Your help is so valuable to this important study. You might remember that last fall/winter you gave us permission to play some games with your child, like naming pictures, counting objects, and sorting cards.

We are planning to play similar games with your child again this spring. The games will take place during the day at your child’s early care and education center and may take up to 45 minutes. Your child will get stickers to thank him/her for playing the games.

Remember that participation is completely voluntary. Please let us know if you or your child would like to stop participating.

As always, if you have any questions about your child’s participation in this project, please call or email [ADD STUDY CONTACT INFO]. This study is being conducted by a team of staff from MDRC, Abt Associates, Frank Porter Graham Child Development Institute, and MEF Associates.

Thank you again!

Sincerely,

The VIQI Study Team

ASSESSMENT PROTOCOL

Assessment Procedures: On the day of direct child assessments, observers will meet with teachers to explain the assessment procedures, using the talking points listed below. Assessors will ask teachers to introduce them to the children being assessed in the classroom. Assessors will make small talk with children in the classroom, beginning to build rapport, before bringing them to the assessment room (a predetermined spot in the classroom where assessments can be conducted without interruption or distractions).Throughout the assessment, assessors will offer children stickers and breaks as the child needs them.

Teacher Talking Points:

• Overview: We will conduct child assessments with selected children in your classroom who have consented to participate in the VIQI Study. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-XXXX and the expiration date is XX/XX/XXXX.

• Assessment Purpose: These assessments will be used to help us build evidence about how to best improve the quality of early child care and education.

• Assessment Experience: The assessments are structured as games, similar to ‘Simon Says.’ Children will be asked to do things like sort pictures and count objects.

• Assessment Length: The assessments will take an hour, at most, for each child in the study.

• Privacy: The research team will do everything possible to keep the identity of the children and classroom private, to the extent permitted by law. Only the research team will view the assessment data. Child and classroom names will never be named in reports or other publications or presentations.

• Voluntary: The child assessments are voluntary. The child and his or her parents choose whether to participate or not, and may opt out at any point in the study. By being on my list for assessments, this indicates that the parent or guardian already gave permission for this child to participate in assessments.

Example Child Assessment Introduction Script: Before administering the child assessment, assessors will briefly explain what the activities will entail. The script assessors will use is provided below. This explanation will not only tell the child what the assessor will be doing and that some of the activities may be difficult, but it will also be a way of obtaining the child’s assent. The assessor will use the following script at the start of assessments to introduce the process to children:

“Hello, [CHILD NAME], my name is [ASSESSOR NAME]. Today, we will be playing some games. I will have some pictures to show you and some things I will ask you to do. Please listen carefully and do the best you can. Some of the things I will ask you to do are hard, even for older kids. So, don’t worry if you’re not sure about them. Just give it your best try. I have some stickers here for you when we are finished with the different parts. Are you ready to begin?”

Reluctant Respondents: We expect most children to be friendly and willing to cooperate, but it is expected that some may be hesitant or fearful of the process. Assessors will be trained to become skilled at identifying children’s reluctance before it escalates. When put into an assessment situation, some children may become stressed or anxious. Others may leave the table to run around. By being attentive to the child who is slow to warm up or easily distracted, assessors can potentially avoid a situation in which they are unwilling to continue. Assessors will learn to recognize when a child may need a break and that by stopping for a few minutes, the child may become re-engaged in the assessment and not end up refusing. Some children may just be having a bad day, so returning at another time may be an appropriate option.

Example Child Assessment End Script: Assessors will end the assessment by thanking children and giving them stickers. The assessor will use the following script at the end of assessments to thank children for their hard work:

“Congrats – we’re all finished! You did a great job. Thank you so much for your hard work! I have some stickers here for you. Please pick a few, and then I will bring you back to your classroom.”

FOLLOW-UP ASSESSMENT BATTERY

The team will administer up to four of direct assessments at follow-up. Examples of assessments might include:

• The Learning Express (McDermott et al., 2009), a multiform assessment that captures young children’s alphabet knowledge, vocabulary, listening comprehension and mathematics skills.

• Test of Early Mathematics Ability (TEMA-3) (Ginsburg & Baroody, 2003) measures young children’s math skills.

• Preschool Language Scales, 5^th Edition (PLS) and the PLS Spanish (Zimmerman, Steiner, & Pond, 2011; Zimmerman, Steiner, & Pond, 2012) assess language development in children ages birth to 7:11 with Auditory Comprehension and Expressive Communication subscales.

• Lens on Science (Greenfield, 2015) measures young children’s science knowledge and nature.

• Pencil Tap (Diamond & Taylor, 1996) is a measure of executive functioning that captures inhibitory control, working memory, and attention.

• Woodcock Johnson III Letter-Word (Gormley et al., 2005; Jordan et al., 2002) and the Spanish version, Bateria III Woodcock-Munoz Ident de letras y palabras (McGrew et al., 2007; Schrank et al., 2005), measures children’s letter and word identification skills.

• Preschool Self-Regulation Assessment (PSRA) (Smith-Donald, Raver, Hayes, and Richardson 2007) is an assessor report that assesses children’s self-regulation in the emotional, attentional, and behavioral domains. The PSRA is completed by assessors about the child after the child finishes the general battery of assessments.

3