State Child Nutrition Agency (State Director and Key Staff)

OMB Number: 0584-NEW Expiration Date: XX/XX/XXXX

APPENDIX C-4. PRETEST: SITE VISIT INTERVIEW GUIDE

Assessing the Child Nutrition State Administrative Expense Allocation Formula (Child Nutrition SAE)

Pretest of the Site Visit Interview Guide

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-xxxx. The time required for the State Child Nutrition Director to provide this information collection is estimated to average 1.5 hours per response, including the time to review instructions, search existing data resources, gather and maintain the data needed, and complete and review the collection of information.

Introduction

Good morning/afternoon. Thank you again for taking the time to talk with me today. My name is [INTERVIEWER NAME] and as you know, I work for Westat, a private research company in Rockville, Maryland.

The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) is trying to better understand spending patterns and factors that affect State use of State Administrative Expense (SAE) funds. FNS hired Westat to conduct the Child Nutrition SAE study to assess the effectiveness of State’s current formulas, and understand what influences States’ spending allocation decisions. The study will include interviews with State agency Directors in 12 States to understand States’ processes for budgeting and spending SAE funds. If there are multiple state agencies that receive SAE funds in a State we will conduct interviews with staff from all of them. Before we conduct these interviews on a broader scale, we are interested in identifying areas for improving the questions – the goal is to ensure that the questions are easy to understand and to answer.

Did you receive a copy of interview questionnaire?

IF YES: Do you have that in front of you at the moment? If not, could you please get the copy because we’ll need to review it while we’re talking today.

IF NEVER RECEIVED OR LOST: Are you able to access your email?

HAS ACCESS BUT NO EMAIL ADDRESS ON FILE: I can resend right now via email. Can you please give me your email address?

HAS ACCESS TO EMAIL: I’ve resent the documents.

IF NO ACCESS TO EMAIL EVER, OFFER TO RESEND AND RESCHEDULE INTERVIEW. IF CANNOT RESCHEDULE, PROCEED WITH INTERVIEW: I will mail you another copy of the document to you. Let me confirm your address.

Informed Consent

Before we get started there are a few things I want to mention.

• This is a research study which means your participation in this discussion is voluntary.

• Refusal to participate will not have any impact on your position or child nutrition programs.

• The thoughts and opinions that you share during our discussion will be used for research purposes only. They will not be used as an evaluation of your work, your staff’s work, or the Child Nutrition program.

• There are no direct benefits to you or your department for participating in this study.

• Your participation and feedback is important and needed because input from staff like yourself help us refine our instruments to best explore the SAE in different states.

• The names of those who participate in these cognitive tests of the interview guides may be shared, and we may quote you if we feel a comment is particularly poignant. However, once we start the interview, you can take a break, skip questions, choose to make comments off the record, or stop participating at any time. Again, there will be no penalty or loss of benefits to your program.

• Finally, with your permission, I would like to record this discussion. The recording helps us recall exactly what was said when we go to summarize our findings. The recordings and any notes we have will be stored on Westat’s secure server and will be destroyed after the project is complete.

• If you have any questions about your rights as a participant, please call the Westat Human Subjects Protections office at 1-888-920-7631. Please leave a message with your full name, the name of the research study that you are calling about (Child Nutrition SAE Study), and a phone number beginning with the area code. Someone will return your call as soon as possible.

We have planned for this discussion to last about 90 minutes.

Do you have any questions? [ANSWER ALL QUESTIONS]

Do I have your consent to participate?

May I turn on the audio recorder now?

IF YES: [ONCE RECORDER IS ON, “Now that the recorder is on, today is DATE at TIME. Do you still consent to participate in the testing of this interview guide, and do I have your permission to audio record this interview?” MAKE SURE YOU GET AN AUDIBLE “Yes” FROM RESPONDENT.]

Procedures

First I’ll ask you questions about your State’s SAE budgeting and spending patterns, and then dive into questions about factors that influence SAE allocations and how specific Child Nutrition programs impact SAE. After the interview is over, I’ll ask you some debriefing questions at the end to get your reactions to the overall content, wording, and flow of the interview questions.

ADMINISTER IN-DEPTH INTERVIEW. NOTE BEGIN TIME: __________________

NOTE DELIVERY, COMPREHENSION AND RESPONSE ISSUES ON TRACKING DIRECTLY ON INTERVIEW GUIDE.

NOTE ANY VERBAL REACTION R SHARES FOR FOLLOW-UP DURING THE DEBRIEF.

FOR ANY NON-VERBAL REACTIONS (E.G., LONG PAUSES), ASK ONLY, Tell me what you’re thinking here. DO NOT PROBE BEYOND THAT UNTIL THE DEBRIEF.

COGNITIVE TESTING RESEARCH QUESTIONS TO REMEMBER:

• Do respondents understand and interpret the questions as intended?

• Are State Directors aware of who has a role in budgeting and spending SAE?

• Are State Directors able to answer in-depth questions about SAE Spending Patterns, including budgeting and spending SAE funds, the services or activities that the SAE funds, and the process and reasons to reallocate or return SAE funds?

• Are State Directors able to answer in-depth questions about the factors that influence the State’s use of SAE?

• Are State Directors able to answer questions about SAE funds that pertain to specific Child Nutrition programs?

• Is there any information about the data collection and submission process that is missing from the in-depth interview guide?

Site Visit Interview Guide

Warm Up:

1. To begin, please tell me about your current role and how long you have worked here. [IF INTERVIEW IS WITH STATE DIRECTOR, RECAP THE INFORMATION HE/SHE PROVIDED IN THE PRE VISIT TELEPHONE INTERVIEW AND MOVE TO SECTION II.]

II. State Administrative Expense (SAE) Spending Patterns

[AS APPROPRIATE, REFER TO COPY OF STATE SAE PLAN AND OTHER BUDGET DOCUMENTS PROVIDED BY THE STATE AFTER THE PRE VISIT INTERVIEW.]

I would first like to discuss some of the specifics regarding your SAE planning process.

2. Please walk me through the process, step-by-step, that your agency uses for developing the SAE budget and spending plan.

• Probes:

  • How does the process begin?

  • Who is involved?

  • What is the timing of each of the steps? Does it vary much from year to year?

  • How do you identify needs?

3. What activities and services are SAE funds spent on? Please walk me through each of the main categories in your SAE budget.

• Probes:

  • How are funds allocated across the different CN programs that your agency administers?

  • Are there program areas that you feel need more funding? Why?

  • How does planned spending compare to actual spending? Has that varied over the last decade or so?

  • How does this compare to the SAE Plan that you submitted to FNS?

4. Next I’d like to discuss the reallocation process and your State agency’s use of reallocated funds. Based on the information we have from FNS – your agency received reallocated funds [NUMBER] times over the past 10 years. Why did you request funds through reallocation? OR Why did you not request reallocated funds?

• Probes (can be used to probe each instance mentioned):

  • Why were additional funds requested?

  • What activities or services were the additional funds used toward?

  • Could you walk me through the decision-making process for requesting reallocation funds? Who are the key players involved in the decision-making process?

  • What challenges did you face with the reallocation process?

    • What strategies have you used to overcome these challenges?

5. We are also interested in circumstances surrounding the return of unused SAE funds. Our information also shows that you have had [NUMBER] instances of returning SAE funds over the past decade. Can you discuss these experiences?

   • Probes (can be used to probe each instance):

     • Why wasn’t the entire allocation used?

     • At what point in time did you know that you would not spend the entire allocation?

6. [IF FUNDS HAVE NEVER BEEN RETURNED] Our information also shows that your State agency has never returned SAE funds. What aspects of your SAE budgeting and planning process do you think contributed to this?

• Probes:

  • What strategies would you suggest other States employ to have similar outcomes?

7. As you know, State agencies may carry over up to 20 percent of their initial SAE allocation to the next fiscal year. According to the information we have, you carried over funds in [LIST FY’s]. Please describe how carryover helps you effectively manage your SAE funds?

   • Probes:

• How does the carryover flexibility influence your budgeting and planning process?

• What challenges, if any, do you have with the carryover restrictions?

8. [ASK ONLY OF STATES WITH MULTIPLE CN AGENCIES] States are also permitted to transfer SAE funds within the State to other agencies that receive SAE. Your State has done this [X] times in the recent past.

[IF THE STATE HAS TRANFERRED FUNDS:] Can you walk me through your experience with funds transfer?

• Probes:

• How did the funds transfer come about?

• Did the FNS regional office facilitate the transfer?

• What process did you use to transfer funds?

• Are there any State policies on funds transfer? Describe and explain.

[IF THE STATE HAS NOT TRANSFERRED FUNDS:] Have you ever considered funds transfer? Why or why not?

• Probes:

  • Has the FNS regional office discussed funds transfer with your agency? What were the circumstances?

  • Are there any State policies on funds transfer? Describe and explain.

III. Factors Affecting State Use of SAE Allocations

Next I want to talk about some of the factors that may affect your agency’s use of SAE.

9. First let’s talk about when funds are made available by FNS. How does the timing of the provision of funds by FNS impact your use of the funds?

• Probes:

  • What challenges have you faced due to timing?

    • …related to when the initial SAE allocation is provided?

    • …related to expending funds within the required timeframe?

  • How about the timing of reallocation requests?

    • How has the timeline influenced your requests for reallocated funds?

    • What strategies have you used to obligate reallocated funds between receipt and the end of the FY?

    • What challenges have you faced with this process?

  • What strategies have you used to overcome these challenges?

  • What benefits does the SAE allocation and reallocation timeline have on the budgeting and planning process?

10. What other factors would you say impact your agency’s ability to use SAE? This impact could be negative or positive.

    • Probes:

      • Number of [SFAs/institutions/recipient agencies]?

      • State geography and travel distances?

      • Number of State agencies receiving SAE?

11. How efficient do you believe the current SAE allocation is in meeting your current and changing administrative needs?

[INTERVIEWERS SHOULD BE PREPARED TO ASK QUESTIONS ABOUT SPECIFIC INSTANCES THROUGHOUT THE PAST DECADE, INCLUDING ITEMS RESPONDENTS DO NOT BRING UP THEMSELVES.]

• Probes:

  • Do you generally feel your agency has enough SAE to operate effectively? How do you handle new priorities that arise (e.g., new legislative/FNS mandates)?

  • What impact have you seen on staffing levels?

12. [ASK IF NEEDED BASED ON RESPONSES PROVIDED IN PRE-VISIT INTERVIEW] How does the State environment affect SAE budgeting and spending?

• Probes:

  • How does State legislature affect SAE budgeting and spending?

  • How do State executive mandates influence SAE budgeting and spending?

  • What role does advocacy have in SAE budgeting and spending?

  • How does allocation of the entire State budget influence SAE budgeting and spending?

• [WITH REGARD TO ANY MENTIONED CHALLENGES ASSOCIATED WITH THE PROBES] What strategies have you used to overcome these challenges?

• How has your unique State environment [CAN REFERENCE SPECIFIC ITEMS ABOVE] helped your programs?

13. Your State’s funding requirement for CN Programs is [X], and your State contributed [Y]. Tell me more about how this funding level is determined in your State.

    • Probes:

      • Is there any legislation or documentation available about your state’s funding requirement?

      • What are the funds used for?

      • Have there been changes in the amount provided? Please explain.

14. What impact would you say technology has had on the use of SAE funds? Levels?

• Probes:

  • How does in the introduction of new technology influence SAE budget/planning?

  • How has technology helped your programs?

  • To what extent does technology impact staffing needs?

15. Are there any other challenges or barriers we haven’t discussed? What about any notable successes you would like to mention?

    • Probes:

      • What do you think your agency does particularly well with regard to managing SAE funds?

IV. Alternatives to the Current Formula

Now I would like to ask for your input on the current formula used by FNS to allocate SAE funds to State agencies. [PROVIDE RESPONDENTS WITH A COPY OF ‘DESCRIPTION OF SAE ALLOCATION PROCESS’ AS A REFERENCE. WALK THE RESPONDENT THROUGH THE FORMULA, AS NEEDED.]

16. Seeing how the current SAE allocation formula is structured, what aspects of the formula would you change? Why? (The total amount of SAE funding would remain the same.)

    • Probes:

      • How would those changes allow you to use your funds more efficiently?

      • Who would benefit the most from these changes? Who do you think would benefit the least?

V. Program-Specific Questions

NOTE: These will be asked based on which program(s) the State agency administers.

I would like to ask you a few questions about the specific Child Nutrition (CN) programs your agency administers, and how they impact use of SAE.

School Meal Programs:

The school meal programs for SAE purposes are the National School Lunch Program (NSLP), the School Breakfast Program (SBP), and the Special Milk Program (SMP).

17. [IF AGENCY THAT ADMINISTERS FDP IS SEPARATE AGENCY:] Could you tell me a little about the communication and/or coordination efforts between the separate agencies that administer School Meals and USDA Foods?

• Probes:

• How do your agencies communicate about available funds and/or transfer of funds to support USDA Foods activities?

• What collaboration is there about program issues?

• What influence has this had on the FDP? On the school programs?

1. [IF SAME AGENCY ADMINISTERS SCHOOLS AND FDP:] What impact do you think there is of having the FDP for schools administered by your agency?

   • Probes:

     • Are there efficiencies through the single administration?

     • Are there challenges?

2. How did your agency utilize Healthy, Hunger-Free Kids Act (HHFKA) Section 201 State funds?

• Probes:

  • What benefits/challenges did you face in using these funds?

  • What impact did it have on your use of SAE funds? How?

  • How did it influence your administration of the school meal programs?

20. How has implementation of a new Administrative Review process impacted your use of SAE?

• Probes:

  • Did staffing for reviews change?

  • Other impacts or challenges?

Child and Adult Care Food Program:

Now I would like to know a little about the administration of the Child and Adult Care Food Program (CACFP).

21. How many CACFP agreements does your agency currently manage?

• Probes:

  • What factors influence your spending of SAE funds towards CACFP?

  • How does this influence your budgeting and spending towards this program?

22. [IF ADULT CARE IN SEPARATE STATE AGENCY:] Could you tell me a little about the communication and/or coordination efforts between the separate agencies that administer the child and adult portions of CACFP?

• Probes:

  • How do your agencies communicate about available funds and/or transfer of funds?

  • What collaboration is there on program issues?

  • What impact does this separate administration have on use of SAE?

Food Distribution for Schools:

Finally, I’d like to talk about your State’s administration of the Food Distribution Program for Schools (FDP).

23. [IF AGENCY THAT ADMINISTERS SCHOOLS IS SEPARATE AGENCY:] Could you tell me a little about the communication and/or coordination efforts between the separate agencies that administer School Meals and USDA Foods?

• Probes:

• How do your agencies communicate about available funds and/or transfer of funds to support USDA Foods activities?

• What collaboration is there about program issues?

• What influence has this had on the FDP? On the school programs?

24. What technological investments has your agency made for the FDP?

• Probes:

  • Has the WBSCM been deployed to local agencies?

    • What benefits/challenges have you faced with this system?

    • Other technology?

VI. Interview Wrap-Up

Thank you so much for taking the time to talk with us today. I’d like to wrap up our discussion with a few final questions.

25. Reflecting on everything we have talked about, what do you think your State does really well with regard to SAE budgeting and spending?

26. Is there anything you believe we should have discussed today but haven’t? Is there anything we missed?

27. Do you have any final questions or thoughts?

Thank you again. Have a wonderful day!

Debriefing Questions

NOTE END TIME: _________________________

Thank you. Now I’m going to ask you some questions about the interview overall. First, tell me your overall impressions of the questions I just asked you.

USE RELEVANT PROBES BELOW TO DISCUSS OBSERVED ISSUES FROM INTERVIEW NOTES.

Clarification Requests

You asked [FILL] for question XX. Can you say more about what you were thinking there?

Hesitation

Can you say more about your hesitation at question XX?

Confusion

What was it about question XX that seemed confusing to you?

IF NEEDED, In your own words, what information would you say [CONFUSING TEXT] is asking for?

Answer Changes

What made you decide to change your answer at question XX?

If R said question did not apply

For question XX you said it did not apply. Can you say more about that?

If R could not answer

For question XX, you said you couldn’t answer. Can you say more about that?

IF NEEDED:

• Is the information collected or tracked in your state? IF YES, Who has access to it?

Wrap-up

Is there anything else from your earlier review of the questions that we have not talked about today?

Which questions did you think were hardest to answer? What makes you say that?

Which questions did you think were easiest to answer? What makes you say that?

Those are all the questions I have for you. Is there anything we haven't discussed that you would like to mention?

Thank you for your time.

STOP RECORDING.

Draft Instruments	0