State, Local and Tribal Agencies (Directors, SFA & Schools)

APPENDIX C-3. STATE CHILD NUTRITION KEY STAFF PRE-TEST PROTOCOL

OMB Number: 0584-NEW

Expiration Date: XX/XX/XXXX

School Meals Forms Study

Key Staff Guide

Cognitive Interview Protocol

Introduction

Good morning/afternoon. Thank you again for taking the time to talk with me today. My name is [INTERVIEWER NAME] and as you know, I work for Westat, a private research company in Rockville, Maryland and we are contacting you on behalf of United States Department of Agriculture (USDA), Food and Nutrition Service (FNS).

The US Department of Agriculture’s, Food and Nutrition Service (FNS) is trying to better understand and improve how school meal program data is collected and reported. FNS has hired Westat to help conduct the School Meals Forms Study to look at how schools, School Food Authorities (SFAs), and State agencies collect and report school meal program data through USDA FNS forms. Part of this study includes talking directly with State Child Nutrition staff to better understand data collection and reporting for school meal program forms. Before we conduct these interviews on a broader scale, we are interested in identifying areas for improving the questions – the goal is to ensure that the questions are easy to understand and to answer.

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-NEW. The time required for the State Child Nutrition Director to provide this information collection is estimated to average 60 minutes per response, including the time to review instructions, search existing data resources, gather and maintain the data needed, and complete and review the collection of information.

Did you receive a copy of interview questionnaire and FNS forms?

IF YES: Do you have that in front of you at the moment? If not, could you please get the copy because we’ll need to review it while we’re talking today.

IF NEVER RECEIVED OR LOST: Are you able to access your email?

HAS ACCESS BUT NO EMAIL ADDRESS ON FILE: I can resend right now via email. Can you please give me your email address?

HAS ACCESS TO EMAIL: I’ve resent the documents.

IF NO ACCESS TO EMAIL EVER, OFFER TO RESEND AND RESCHEDULE INTERVIEW. IF CANNOT RESCHEDULE, PROCEED WITH INTERVIEW: I will mail you another copy of the document to you. Let me confirm your address.

IF NO ACCESS TO EMAIL DURING THE CALL: Okay, I will send you the documents again. We will need to find another time to schedule the interview so you can have the application in front of you during our conversation.

Informed Consent

Before we get started there are a few things I should mention.

• This is a research study which means your participation in this discussion is voluntary.

• Refusal to participate will not have any impact on your position as State Child Nutrition staff.

• The thoughts and opinions that you share during our discussion will be kept private and your name will not appear in the main report or any documentation shared with FNS, the school district, or the State Child Nutrition department.

• The thoughts and opinions that you share during our discussion will be used for research purposes only. They will not be used as an evaluation of your work, your staff’s work or the Child Nutrition program.

• There are no direct benefits to you or your Child Nutrition program for participating in this study.

• Your participation and feedback is important and needed because input from staff like yourself help us better understand how reporting can be improved.

• Once we start the interview, you can take a break, skip questions or stop participating at any time. Again, there will be no penalty or loss of benefits to your State Child Nutrition program.

• Finally, with your permission, I would like to record this discussion. The recording helps us recall exactly what was said when we go to summarize our findings. The recordings and any notes we have will be stored on Westat’s secure server and will be destroyed after the project is complete.

We have planned for this discussion to last no more than an hour.

Do you have any questions? [ANSWER ALL QUESTIONS]

Do I have your consent to participate?

May I turn on the audio recorder now?

IF YES: [ONCE RECORDER IS ON, “Now that the recorder is on, today is DATE at TIME. Do I still have your permission to audio record this interview?” MAKE SURE YOU GET AN AUDIBLE “Yes” FROM RESPONDENT.]

Procedures

First I’ll ask you questions about meal reporting forms. After that part of the interview is over, I’ll ask you some debriefing questions at the end to get your reactions to the questions about meal forms.

ADMINISTER IN-DEPTH INTERVIEW. NOTE BEGIN TIME: __________________

NOTE DELIVERY, COMPREHENSION AND RESPONSE ISSUES ON TRACKING DIRECTLY ON INTERVIEW GUIDE.

NOTE ANY VERBAL REACTION R SHARES FOR FOLLOW-UP DURING THE DEBRIEF.

FOR ANY NON-VERBAL REACTIONS (E.G., LONG PAUSES), ASK ONLY, Tell me what you’re thinking here. DO NOT PROBE BEYOND THAT UNTIL THE DEBRIEF.

RESEARCH QUESTIONS

• Do respondents understand and interpret the questions as intended?

• Are key CN staff aware of how data is calculated or managed for the forms?

• Are key CN staff able to answer in-depth questions about how the State communicates with SFAs, schools, and sites?

• Are key CN staff able to answer in-depth questions about communication with FNS regional offices?

• Are key CN staff able to answer in-depth questions about errors, delays or innacuracies in the process?

• Is there any information about the data collection and submission process that is missing from the in-depth interview guide?

Debriefing Questions

NOTE END TIME: _________________________

Thank you. Now I’m going to ask you some questions about it. First, just tell me your overall impressions of the questions I just asked you about school meals forms.

USE RELEVANT PROBES BELOW TO DISCUSS OBSERVED ISSUES FROM INTERVIEW NOTES.

ASK ABOUT THE ISSUES IN THE ORDER THEY OCCURRED.

REPEAT QUESTION TEXT TO REORIENT R.

Clarification Requests

You asked [FILL] for question XX. Can you say more about what you were thinking there?

Hesitation

Can you say more about your hesitation at question XX?

Confusion

What was it about question XX that seemed confusing to you?

IF NEEDED, In your own words, what information would you say [CONFUSING TEXT] is asking for?

Answer Changes

What made you decide to change your answer at question XX?

If R said question did not apply

For question XX you said it did not apply. Can you say more about that?

If R could not answer

For question XX, you said you couldn’t answer. Can you say more about that?

IF NEEDED:

• Is the information collected or tracked in your district? IF YES, Who has access to it?

Wrap-up

Is there anything else from your earlier review of the questions that we have not talked about today?

Which questions did you think were hardest to answer? What makes you say that?

Which questions did you think were easiest to answer? What makes you say that?

Closing and Incentive

IF OBSERVERS ARE PRESENT, CHECK TO SEE IF THEY HAVE FURTHER QUESTIONS.

Those are all the questions I have for you. Is there anything we haven't discussed that you would like to mention?

DISCUSS ANY RESPONDENT COMMENTS.

Thank you for your time.

STOP RECORDING.

5