Attachment A - Example Instruments

Attachment A: example instruments approved under the pretesting generic CLEARANCE (0970-0355)

Contents

Gen IC: Measurement Development: Quality Relationship of Family and Family Services Staff in Head Start/Early Head Start – Staff Measure 2

Gen IC: Redesign of the Head Start Family and Child Experiences Survey (FACES 2012) – Cognitive Interview Protocol 15

Gen IC: Measurement Development: Quality Relationship of Family and Family Services Staff in Head Start/Early Head Start

Instrument: Family Services Staff Measure

OMB No.: 0970-0355

Expiration date: 01/31/2015

Family and Provider/Teacher Relationship Quality

Family Services Staff Measure

Public reporting burden for this collection of information is estimated to average 15 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number.

The office of Management and Budget has approved the data collection under OMB #0970-0355. OPRE is authorized to conduct this study under Section 649 of the Head Start Act, as amended by the Improving Head Start for School Readiness Act of 2007, codified at 42 United States Code (U.S.C.) 9844.

By Family Service Worker (FSW) we mean someone who helps families identify their goals for themselves and their child; connect families to resources and services that support the family and the child; and help families advocate for themselves. FSW are also known by many different names and titles; some examples include Family Services Staff, Family Advocates, Home Visitors, and Family Services Coordinator. The term Family Services Staff is used in all materials related to this measure.

We would like to learn about how you and the families in your program work together.

1. Since September, how many of the families you serve have you directly helped in any of the following ways:

[MARK ONE BOX IN EACH ROW.]

	None	Some	Most	All
a. Encouraged families to seek or receive services?
b. Followed up with families about whether services they have received met their needs?
c. Made appointments or arrangements for families to receive services they need?
d. Helped families find services they need?
e. Advocated on behalf of families to ensure that outside service providers are responsive?

2. Since September, how often have you been able to do the following?

[MARK ONE BOX IN EACH ROW.]

	Never	Rarely	Sometimes	Very often
a. Followed up with parents about goals they set for their child
b. Followed up with parents about goals they set for themselves
c. Offered parents ideas or suggestions about parenting
d. Suggested activities for parents and children to do together
e. Worked with parents to develop strategies they can use at home to support their child’s learning and development
f. Taken parents’ values and culture into account when serving them
g. Offered parents books and materials on parenting

3. Thinking about the families you serve, how many parents have you met with or talked to about the following?

[MARK ONE BOX IN EACH ROW.]

	None	Some	Most	All
a. How many children they have
b. How many adult relatives live in their households
c. Their work and school schedules
d. Their marital status
e. Their parenting styles
f. Their employment status
g. Their family’s financial situation
h. The role that faith and religion play in their household
i. Their family’s cultures and values
j. What they do outside of the Head Start setting to encourage their children’s learning
k. How they discipline their children
l. Problems their child is having at home
m. Changes happening at home
n. Health issues their children may have
o. Health issues they or other family members may have

4. Please indicate how much you agree or disagree with each of these statements.

[MARK ONE BOX IN EACH ROW.]

	Strongly disagree	Disagree	Agree	Strongly agree
a. My goal is to help parents reach their full potential
b. I help parents to reach their job and educational goals
c. I work with parents to figure out the steps to reach their goals
d. I encourage parents to make decisions about their children’s education and care
e. Parents’ beliefs about childcare and education vary by culture
f. I encourage parents to provide feedback on the services and support I provide them
g. I am open to using information on different ways to help parents and children
h. When it comes to their children, parents are the experts
i. Even though my professional or moral viewpoints may differ, I accept that parents are the ultimate decision makers for the care and education of their children

5. Please indicate how much you agree or disagree with these statements.

[MARK ONE BOX IN EACH ROW.]

	Strongly disagree	Disagree	Agree	Strongly agree
a. Sometimes it is hard for me to support the way parents raise their children
b. Sometimes it is hard for me to support the way parents discipline their children
c. Sometimes it is hard for me to accept the different cultural beliefs of parents
d. Sometimes it is hard for me to support the goals parents have for their children
e. Sometimes it is hard for me to work with parents who have different beliefs than me
f. Sometimes it is hard for me to accept the choices that parents make

6. When providing services to families in your program, how often do you take into account the following?

[MARK ONE BOX IN EACH ROW.]

	Never	Rarely	Sometimes	Very often
a. Information parents share about their child
b. Whether activities are welcoming to all family members, including fathers
c. Information parents share about their home life
d. What you can do to make fathers or other family members feel comfortable at centers
e. Families’ values and cultures
f. Information parents share about their career or education goals
g. Information parents share about their “life goals”

7. Since September, how often have you met with or talked to parents about the following?

[MARK ONE BOX IN EACH ROW.]

	Never	Rarely	Sometimes	Very often
a. How their child is doing in the Head Start/Early Head Start program
b. Their child’s learning or development
c. Goals parents have for their child
d. Goals parents have for themselves
e. How parents are progressing towards goals they have for themselves
f. Problems their child is having in the Head Start/Early Head Start program
g. Problems parents may be having with their work or school
h. Parents’ vision for their family’s future

8. People vary in what they consider part of their job. Please indicate how much you agree or disagree with the following statements.

Part of my job is to…

[MARK ONE BOX IN EACH ROW.]

	Strongly disagree	Disagree	Agree	Strongly agree
a. Help families get services available in the community
b. Offer parents information about community events
c. Respond to issues or questions outside of my normal work hours
d. Learn the values and beliefs of the families I serve
e. Change my work schedule in response to parents’ work or school schedules
f. Learn new ways to assist families
g. Change how services are offered to children and families in response to parent feedback
h. Talk to parents about parenting
i. Help parents reach their goals
j. Tailor my approach when working with mothers, fathers, or other family members
k. Help parents learn skills needed to succeed
l. Consider how culture shapes the way I should approach my work with families
m. Make home visits to provide support and to work on goal setting with the families
n. Help families meet their basic needs

9. If families have a question or a problem comes up during the day, how easy or difficult is it for them to reach you?

[MARK ONE BOX IN EACH ROW.]

Very difficult

Difficult

Easy

Very easy

10. Since September, how many of the families you serve have you given information on the following:

[MARK ONE BOX IN EACH ROW.]

	None	Some	Most	All
a. Employment or job training?
b. Food banks or pantries?
c. Child care subsidies or vouchers?
d. Adult education, GED classes, ESL classes, or continuing education?
e. Housing assistance?
f. Energy or fuel assistance?
g. Parenting skills group?
h. Health insurance ?

11. Since September, have you provided referrals for the following services, within your agency or the community:

[MARK ONE BOX IN EACH ROW.]

	Yes, I made a referral	No, I did not make a referral	Not applicable
a. Health screening for children (medical, dental, vision, hearing, or speech)?
b. Developmental assessments for children?
c. Counseling services for children?
d. Counseling services for parents?
e. Social services such as housing assistance, food stamps, financial aid, or medical care?
f. Nutritional screening for children?
g. Legal services?
h. Substance abuse?
i. Crisis assistance?
j. Domestic violence?

12. People work as Family Service Workers for many reasons. Please indicate how much you agree or disagree with the following statements:

[MARK ONE BOX IN EACH ROW.]

	Strongly disagree	Disagree	Agree	Strongly agree
a. I work as a Family Service Worker because I enjoy it
b. I see this job as just a paycheck
c. I work as a Family Service Worker because I like helping families reach their goals
d. If I could find something else to do to make a living I would
e. I work as a Family Service Worker because I like helping children and families get the services they need

13. In the last ten years, have you received training or coursework on how to recognize signs of:

[MARK ONE BOX IN EACH ROW.]

	Yes	No
a. Child abuse and neglect
b. Domestic violence
c. Substance abuse
d. Depression or mental health issues in parents
e. Hunger
f. Developmental delays in children
g. Developmental delays in adults

14. How many families do you currently serve?

_________________ families

15. How many centers do you currently serve?

_________________ centers

16. How many years have you been working in this field?

_________________ years

17. How long have you worked at your current center(s)?

_________________ years

18. Do you have children living in your household who attend Head Start/Early Head Start now?

[MARK ONLY ONE BOX.]

Yes

No

19. Did you ever have a child in your household who attended Head Start/Early Head Start?

[MARK ONLY ONE BOX.]

Yes

No

The next set of questions asks about your background.

20. Are you of Hispanic or Latino origin?

[MARK ONLY ONE BOX.]

Yes

No

21. What is your race?

[MARK ALL THAT APPLY.]

White

Black or African American

American Indian or Alaska Native

Asian Indian

Chinese

Filipino

Japanese

Korean

Vietnamese

Other Asian

Native Hawaiian

Guamanian or Chamorro

Samoan

Other Pacific Islander

22. Do you have a Child Development Associate (CDA) credential?

[MARK ONLY ONE BOX.]

Yes

No

23. Do you have some type of family services credential that supports competency in working with families?

[MARK ONLY ONE BOX.]

Yes

No

Name of Credential: __________________________________

24. What is the highest level of education you have completed?

[MARK ONLY ONE BOX.]

Less than a high school diploma

High school diploma or GED

Some college, no degree

Associate’s degree

Bachelor’s degree

Graduate school degree

Thank you!

Gen IC: Redesign of the Head Start Family and Child Experiences Survey (FACES 2012)

Instrument: Cognitive Interview Protocol for FACES Pilot Study: Survey of Well-Being of Young Children

I. Interviewer Introduction Script

The following scripts should not be read verbatim. You, as the interviewer, need to be familiar enough with the scripts below to introduce the think-aloud process in a conversational manner. Text written in italics is suggested content for you to be thoroughly familiar with in advance. You should project a warm and reassuring manner toward the participant to develop a friendly rapport and should use conversational language throughout.

Hello, my name is [NAME].

Thank you for taking the time to help us today.

I am working with the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (DHHS) on a research study called the Family and Child Experiences Survey Pilot Study. Within the past few weeks you completed the Survey of Well-Being of Young Children (SWYC) [over the phone/on the web]. I have some questions about how you answered the questions on the survey, and about your experience answering them. The things you tell me today will help us make our surveys better in the future. This should take about 30 minutes.

Your participation today is voluntary. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. The OMB control number for this information collection is 0970-0355 and it expires 1/31/2015.

I will be taking notes so that we can use what you tell me today to make these items better. Your information will be kept private. Only people working on the research study will know about your answers and ideas, but your name will not be connected to those answers and ideas. We will not talk with anyone outside the study team, including anyone at your child’s Head Start center. Your contributions today may be used only for research purposes and may not be used for any other purpose except as required by law. I would like to record what we are doing today so that if I miss something when I am writing up my notes I can go back to the recording to make sure I got it right. No one else will hear the recording. Is it okay if I record our conversation?

Do you have any questions before we get started?

If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview. After answering questions and giving further explanation, begin the interview with the first item.

II. Survey Items

1. Developmental Milestones

The survey started with questions about your child’s development, and asked you to tell us how much your child does certain things, like “tells you a story from a book or tv” and “draws simple shapes, like a circle or square.”Your answer choices were not yet, somewhat, and very much.

Have the participant’s answers ready if he or she would like to be reminded of them. Note where the participant chose “somewhat” and “very much”.

These questions are about your child's development. Please tell us how much your child is doing each of these things. If your child doesn't do something any more, choose the answer that describes how much he or she used to do it.

Select one per row

	Not Yet	Somewhat	Very Much
a. Talks so other people can understand him or her most of the time	0 	1 	2 
b. Washes and dries hands without help (even if you turn on the water)	0 	1 	2 
c. Asks questions beginning with "why" or "how" - like "Why no cookie?"	0 	1 	2 
d. Explains the reasons for things, like needing a sweater when its cold	0 	1 	2 
e. Compares things - using words like "bigger" or "shorter"	0 	1 	2 
f. Answers questions like "What do you do when you are cold?" or "when you are sleepy?"	0 	1 	2 
g. Tells you a story from a book or tv	0 	1 	2 
h. Draws simple shapes - like a circle or a square	0 	1 	2 
i. Says words like "feet" for more than one foot and "men" for more than one man	0 	1 	2 
j. Uses words like "yesterday" and "tomorrow" correctly	0 	1 	2 
i. Says words like "feet" for more than one foot and "men" for more than one man	0 	1 	2 
j. Uses words like "yesterday" and "tomorrow" correctly	0 	1 	2 
k. Stays dry all night	0 	1 	2 
l. Follows simple rules when playing a board game or card game	0 	1 	2 
m. Prints his or her name	0 	1 	2 
n. Draws pictures you recognize	0 	1 	2 
o. Stays in the lines when coloring	0 	1 	2 
p. Names the days of the week in the correct order	0 	1 	2 

Participant Probes:

1. I see that you responded “somewhat” to a few items, and I’d like you to tell me more about how you arrived at your answer. Read items to participant. In answering this question, what did “somewhat” mean to you? Why did you choose “somewhat” instead of “very much?”

2. I see that you responded “very much” to a few items, and I’d like you to tell me more about how you arrived at your answer. Read items to participant. In answering this question, what did “very much” mean to you? Why did you choose “very much” instead of “somewhat?”

3. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

4. Were there any questions you did not understand?

5. Is there anything else you’d like to tell me about these questions?

2. Preschool Pediatric Symptom Checklist

The survey next asked questions about your child’s behavior. Read questions to participant. Your answer choices were not at all, somewhat, and very much.

Have the participant’s answers ready if he or she would like to be reminded of them. Note where the participant chose “somewhat” and “very much.”

Does your child…	Not At All	Somewhat	Very Much
a. Seem nervous or afraid?	0 	1 	2 
b. Seem sad or unhappy?	0 	1 	2 
c. Get upset if things are not done in a certain way?	0 	1 	2 
d. Have a hard time with change?	0 	1 	2 
e. Have trouble playing with other children?	0 	1 	2 
f. Break things on purpose?	0 	1 	2 
g. Fight with other children?	0 	1 	2 
h. Have trouble paying attention?	0 	1 	2 
i. Have a hard time calming down?	0 	1 	2 
j. Have trouble staying with one activity?	0 	1 	2 

Is your child…	Not At All	Somewhat	Very Much
a. Aggressive?	0 	1 	2 
b. Fidgety or unable to sit still?	0 	1 	2 
c. Angry?	0 	1 	2 

Is it hard to…	Not At All	Somewhat	Very Much
a. Take your child out in public?	0 	1 	2 
b. Comfort your child?	0 	1 	2 
c. Know what your child needs?	0 	1 	2 
d. Keep your child on a schedule or routine?	0 	1 	2 
e. Get your child to obey you?	0 	1 	2 

Participant Probes:

1. I see that you responded “somewhat” to a few items, and I’d like you to tell me more about how you arrived at your answer. Read items to participant. In answering this question, what did “somewhat” mean to you? Why did you choose “somewhat” instead of “not at all” or “very much?”

2. I see that you responded “very much” to a few items, and I’d like you to tell me more about how you arrived at your answer. Read items to participant. What does “very much” mean to you? Why did you choose “very much” instead of “somewhat?”

3. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

4. Were there any questions you did not understand?

5. Is there anything else you’d like to tell me about these questions?

3. Parent’s Concerns

Next, we asked if you had any concerns about your child. Read questions to participant. Your answer choices were not at all, somewhat, and very much.

Have the participant’s answers ready if he or she would like to be reminded of them. Note where the participant chose “somewhat” and “very much.”

	Not At All	Somewhat	Very Much
a. Do you have any concerns about your child's learning or development?	0 	1 	2 
b. Do you have any concerns about your child's behavior?	0 	1 	2 

Participant Probes:

1. I see that you responded “somewhat” to [one or two], and I’d like you to tell me more about how you arrived at your answer. Read items to participant. In answering this question, what did “somewhat” mean to you? Why did you choose “somewhat” instead of “not at all” or “very much?”

2. I see that you responded “very much” to [one or two] items, and I’d like you to tell me more about how you arrived at your answer. Read items to participant. What does “very much” mean to you? Why did you choose “very much” instead of “somewhat?”

3. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

4. Were there any questions you did not understand?

5. Is there anything else you’d like to tell me about these questions?

4. Family Questions

Next, we asked you some questions about your family. Read questions to participant. Your answer choices were yes and no.

Have the participant’s answers ready if he or she would like to be reminded of them. Note where the participant chose “somewhat” and “very much.”

Alcohol, Tobacco, and Substance Use

	Yes	No
a. Does anyone smoke tobacco at home?	1 	0 
b. In the last year, have you ever drunk alcohol or used drugs more than you meant to?	1 	0 
c. Have you felt you wanted or needed to cut down on your drinking or drug use in the last year?	1 	0 
d. Has a family member's drinking or drug use ever had a bad effect on your child?	1 	0 
e. In the past month was there any day when you or anyone in your family went hungry because you did not have enough money for food?	1 	0 

Participant Probes:

1. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

2. Were there any questions you did not understand?

3. Is there anything else you’d like to tell me about these questions?

4. If web: would you have answered these questions differently if an interviewer had asked them instead of you answering by yourself?

5. If phone: Would you have answered differently had you been taking the survey on your own, such as on paper or on the web?

Depression

Over the past two weeks, how often have you been bothered by any of the following problems?	Yes	No
a. Having little interest or pleasure in doing things?	1 	0 
b. Feeling down, depressed, or hopeless?	1 	0 

Participant Probes:

1. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

2. Were there any questions you did not understand?

3. Is there anything else you’d like to tell me about these questions?

4. If web: would you have answered these questions differently if an interviewer had asked them instead of you answering by yourself?

5. If phone: Would you have answered differently had you been taking the survey on your own, such as on paper or on the web?

Relationship with Spouse/Partner

Lastly, we asked you two questions about how you get along with your spouse or partner. Read first question to participant. Your answer choices for this first question were No tension, Some tension, A lot of tension, and not applicable.

In general, how would you describe your relationship with your spouse/partner?

Select one only

 No tension 0

 Some tension 1

 A lot of tension 2

 Not applicable 3

The second question was…Read second question to participant. Your answer choices were No difficulty, Some difficulty, Great difficulty, and Not applicable.

Do you and your partner work out arguments with…

Select one only

 No difficulty 0

 Some tension 1

 A lot of tension 2

 Not applicable 3

Participant Probes:

1. In general, how easy or difficult were these questions to answer? What about the questions made them easy or difficult to answer? Which questions were most difficult? Why?

2. Were there any questions you did not understand?

3. Is there anything else you’d like to tell me about these questions?

4. If web: would you have answered these questions differently if an interviewer had asked them instead of you answering by yourself?

5. If phone: Would you have answered differently had you been taking the survey on your own, such as on paper or on the web?

III. General Questions

1. Survey Mode

Probes for web participants:

1. Did you have any trouble accessing the survey from the e-mail link?

2. How did you take the survey? On a desktop computer, laptop computer, tablet, or smartphone? Does that device belong to you or to someone else, such as a family member, friend, neighbor, public library, or your child’s Head Start Center?

3. Were the question screens easy or difficult to read? What made them easy or difficult to read?

Probes for phone participants:

1. Did you take the survey on a land line or cell phone?

2. How easy or difficult was it to understand the interviewer?

Probes for all participants:

1. How often do you read your e-mail?

2. Do you own or have access to a:

a. Desktop computer?

b. Laptop computer?

c. Tablet?

d. Smartphone?

3. Are you able to access the Internet on any of these devices? Which ones?

4. Where were you when you took the survey? At home, at work, someplace else?

5. Would you prefer to answer these questions on the web or over the telephone?

6. Is there anything that would have made the survey easier for you to complete?

2. Language (asked of all respondents)

Participant Probe:

1. You completed the survey in [English/Spanish]. How well would you say you speak [English/Spanish]: Very well, well, or not very well? If not very well: were any of the questions difficult to answer?

IV. Debriefing and Thank You for Participation

Thank participant for his/her time.

Before we finish, I’d like to hear any other thoughts you have about the survey.

Is there anything else you would like to tell me about answering the survey questions?

Is there anything you would like to ask me about what we did today? [Answer participant questions]

Thank you for helping us to improve our questionnaire.