Business for Profit

Appendix D

Discussion Guide: Retailer Key Informants

Evaluation of Technology Modernization for SNAP Benefit
Redemption through Online Transactions

Advance Email to Retailer Representative for Site Visit

Dear [CONTACT NAME],

As you may know, Abt, Altarum, and Novo Dia are working together to evaluate the Online Purchasing Pilot, which is authorized under the Agricultural Act of 2014¹ (The Farm Bill). Although this study is mandated and required by Congress, participation in this study is voluntary. Be assured that there will be no penalties if you decide not to respond, either to the information collection as a whole or to any particular question. The USDA Food and Nutrition Service is sponsoring this data collection. As part of the evaluation, our team is conducting interviews with key staff at all of the pilot retailers. The purpose of the interviews is to document the implementation of each retailer’s online purchasing pilot, including key design and policy decisions related to the SNAP online purchasing experience; the process, challenges, and lessons of implementation; and the level of effort.

We would like to visit [RETAILER NAME] during [TIME PERIOD] to conduct these interviews. Over the course of the visit, we’d like to hold interviews with a variety of staff involved in the pilot, including the project manager(s); IT personnel (if applicable); customer service manager(s); and personnel involved in fulfillment, shipping, and delivery of SNAP online customer orders (if appropriate). At this time, we ask that you identify the appropriate staff to participate in the interviews. We are happy to schedule a brief phone call to provide more information about identifying staff to participate. Please also let us know what dates you and your team will be available for us to conduct the visit.

Once the site visit dates have been selected, we’ll be in touch with additional site visit logistics, including a list of interview topics to help you prepare for the visit. In the meantime, please let me know if you have any questions whatsoever. We look forward to our visit with you and your team!

Sincerely,

[NAME]

Public reporting burden for this collection of information is estimated to vary from 2 to 3.5 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services, Office of Policy Support, 3101 Park Center Drive, Room 1014, Alexandria, VA 22302, ATTN: PRA (0584-xxxx*). Do not return the completed form to this address.

Evaluation of Technology Modernization for SNAP Benefit
Redemption through Online Transactions

Advance Email to Retailer Representative for Telephone Interviews

Dear [CONTACT NAME],

As you may know, Abt, Altarum, and Novo Dia are working together to evaluate the Online Purchasing Pilot, which is authorized under the Agricultural Act of 2014² (The Farm Bill). Although this study is mandated and required by Congress, participation in this study is voluntary. Be assured that there will be no penalties if you decide not to respond, either to the information collection as a whole or to any particular question. The USDA Food and Nutrition Service is sponsoring this data collection. As part of the evaluation, our team is conducting interviews with key staff at all of the pilot retailers. The purpose of the interviews is to document the implementation of each retailer’s online purchasing pilot, including key design and policy decisions related to the SNAP online purchasing experience; the process, challenges, and lessons of implementation; and the level of effort.

We would like to schedule phone interviews with [RETAILER NAME] during [TIME PERIOD]. The interviews will include a variety of staff involved in the pilot, including the project manager(s); IT personnel (if applicable); customer service manager(s); and personnel involved in fulfillment, shipping, and delivery of SNAP online customer orders (if appropriate). At this time, we ask that you identify the appropriate staff to participate in the interviews. We are happy to schedule a brief phone call to provide more information about identifying staff to participate.

Once the interview times have been confirmed, we’ll be in touch with additional logistics, including a list of interview topics to help you prepare. In the meantime, please let me know if you have any questions whatsoever. We look forward to our discussions with you and your team!

Sincerely,

[NAME]

Public reporting burden for this collection of information is estimated to vary from 2 to 3.5 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services, Office of Policy Support, 3101 Park Center Drive, Room 1014, Alexandria, VA 22302, ATTN: PRA (0584-xxxx*). Do not return the completed form to this address.

Evaluation of Technology Modernization for SNAP Benefit
Redemption through Online Transactions

Draft Discussion Guide: Retailer Key Informants

Introduction

My name is [X], from [Abt/Altarum/Novo Dia Group]. As you may know, Abt, Altarum, and Novo Dia are working together to evaluate the Online Purchasing Pilot, which is authorized under the Agricultural Act of 2014³ (The Farm Bill). Although this study is mandated and required by Congress, participation in this study is voluntary. Be assured that there will be no penalties if you decide not to respond, either to the information collection as a whole or to any particular question. The USDA Food and Nutrition Service is sponsoring this data collection. My colleagues and I are collecting information from a wide range of stakeholders involved in the pilot in order to learn about the implementation process, the challenges you encountered, and lessons learned. I want to start by thanking you for taking the time to speak with us today. Your perspective and insights on these issues are very helpful.

I expect our conversation will take approximately [2/3/3.5] hours.

Before we begin, I would like to assure you that all of your responses will be kept private to the extent provided by law. You may decline to answer any question.

With your permission, we would like to record the conversation to ensure our notes accurately reflect your responses? Recording is voluntary and not required to participate in part or in whole with this data request. Do I have your permission to record our conversation?

Our report to FNS will not be publicly shared. You, or a representative of [RETAILER NAME] will have an opportunity to review the chapter describing your pilot and identify any information that is proprietary and should not be shared with others. The report will describe the range of responses expressed by staff, and may list the names of organizations who contributed information, but we will not quote you or anyone by name or title. However, because of the relatively small number of organizations participating in the study, there is a possibility that a response could be correctly attributed to you.

Public reporting burden for this collection of information is estimated to vary from 2 to 3.5 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services, Office of Policy Support, 3101 Park Center Drive, Room 1014, Alexandria, VA 22302, ATTN: PRA (0584-xxxx*). Do not return the completed form to this address.

First, do you have any questions for me about the SNAP Online Purchasing Evaluation or what we will be discussing today?

Background on Respondent

1. What is your position at [RETAILER NAME]? What are your day-to-day responsibilities?

2. How long have you worked for [RETAILER NAME]?

3. Describe your role implementing the Online Purchasing Pilot.

Pilot Planning and Development

4. How was your organization involved in planning for the pilot and evaluation?

[Probe:]

• What kinds of staff were involved?

• What parts of the process did you work on (e.g., overall management, website design, testing, implementation)?

5. Describe the steps involved in designing, testing, and implementing the pilot.

6. We would like to know how much staff time was involved in the planning and design.

[Interviewer will refer to Table 1 in the Appendix to guide the response to this question.]

[Probe:]

• How many people were involved in pilot planning and design?

• Did you hire new staff specifically to manage the pilot or any aspect of pilot implementation?

7. Do you use a web service provider to provide eCommerce rather than doing work in-house? If so, please describe their roles.

8. What changes were needed to reprogram your website for EBT acceptance?

[Probe:]

• User account/profile

• Shopping process (e.g., identifying SNAP-eligible items)

• Checkout process (e.g., eliminating sales tax on SNAP-eligible purchases; allowing split tender for purposes of charging for delivery or non-SNAP eligible items)

• Weight estimate limitations

• Prepayment/post-order refund

• Terms of service

• Order management

• Additional security modifications

[Probe:]

• How many people were involved in reprogramming?

• About how much time did it take to reprogram?

• Which changes took the most time to implement?

9. What challenges arose when making internal system changes required to comply with FNS and RFV requirements?

[Probe:]

• How did you address those challenges?

• What changes in process resulted from incorporating the new requirements?

10. What changes to your website were needed to implement the Online Purchasing Pilot and to meet First Data’s requirements?

[Probe:]

• Contracting process

• Connectivity

• Implementation of proprietary message formats

• Integration of PIN entry

• First Data refund process

• Settlement and certification

• Other

[Probe:]

• How many people were involved in reprogramming to meet First Data’s requirements?

• About how much time did it take to reprogram?

• Which changes took the most time to implement?

11. Did the acquisition of Acculynk^TM by First Data affect the pilot? If so, how? Does this change have any implications for the potential expansion of SNAP online purchasing? If so, what are the implications?

Pilot Operations

12. What are the steps in processing SNAP online transactions as opposed to standard credit/signature debit?

[Probe:]

• What is the process for identifying a SNAP cardholder? Does it vary for first-time online shoppers versus returning shoppers?

• At what point during the shopping experience do you ask shoppers if they are using a SNAP EBT card? (e.g., up-front at item selection, at checkout)?

• If up-front, how? With specific pop-up screens, banner notices, etc.?

• If at checkout, how?

13. How do you identify SNAP-eligible items? [Note, this may be a particular issue for non “brick and mortar” retailers.]

14. How do online EBT shoppers identify SNAP-eligible items? At what point are items identified as SNAP-eligible (e.g., are they identified on each specific product page, identified on a separate page that lists only SNAP-eligible items, identified for customers with EBT card on file)?

15. Do you charge any type of fees to online SNAP customers? Do the fees for SNAP customers differ from the fees for non-SNAP customers? What methods of payment do you accept for fees?

[Probe:]

Please describe each relevant fee, and provide the dollar amount for each.

• Delivery

• Ordering

• Convenience

• Membership

16. Are any fees or taxes mandated by the State or local jurisdiction? How do you determine what taxes/fees SNAP online customers pay, particularly when dealing with multiple States/jurisdictions?

[Probe:]

Please describe each relevant tax/fee, and provide the dollar amount (or percent tax) for each.

• Bag fees

• Container deposits

• Sales tax

• Other

17. What challenges have you encountered when customers use multiple forms of payment (i.e., split tender)?

[Probe:]

• How do you handle the delivery fee? (e.g., when all products are SNAP-eligible and the delivery fee is the only cost not paid with SNAP EBT)

• How do you handle tax issues on mixed basket check-outs?

• How do you handle restrictions on cash EBT usage (e.g., alcohol, tobacco, lottery tickets)?

[Probe for “brick and mortar” retailers:]

• Are the challenges with multiple forms of payment for online EBT shopping different from those with in-store electronic cash register systems?

18. What happens when selected SNAP-eligible items are out of stock on the website?

19. Is the cart abandonment rate for SNAP EBT customers higher or lower as compared to your existing online shoppers?

20. What happens when selected SNAP-eligible items are not available at fulfillment?

21. It is our understanding that you offer [delivery, pick-up, or both] to SNAP EBT shoppers. Is this still true? If so, what are customers’ preferred methods? Have you seen a change over time? If so, what do you think is contributing to this change?

[Probe:]

• Home delivery

• Shipping

• Store pick-up by location

• Pickup/delivery at separate (non-store) locations; if so, what type of establishments?

• Other retailers whether authorized to redeem SNAP benefits or not

Customer Support

22. What types of support are available to SNAP EBT shoppers when they have issues with making online purchases or delivery/pick-up? If so, how have you changed your support with the implementation of the Online Purchasing Pilot?

[Probe:]

• New FAQs specific to SNAP EBT shoppers

• Training for customer service staff and store clerks

[Probe:]

• Do you outsource customer service or use in-house customer service staff?

23. What have been the most common questions/inquiries from SNAP EBT shoppers?

[Probe:]

• Problems using EBT cards for online shopping

• Problems using EBT PIN for online shopping

• Questions about eligible items

• Order changes

• Undelivered/missing items

• Problems with returns, refunds, or incorrect balances

• What are the allowable responses or resolutions that staff can provide in response to complaints? (e.g., refer customer to individual store; offer refund to SNAP EBT card; offer discount, gift card, store credit, etc.)

24. Has there been a need to refer SNAP EBT customers to the EBT host processor (i.e., Conduent or FIS) or State SNAP agency for assistance? If so, what type of questions require this referral?

[Probe]:

• Has the State SNAP agency contacted you regarding questions they receive from EBT shoppers making online purchases at your store(s)? If so, how frequently?

25. How do the customer service calls you receive from SNAP EBT shoppers differ from calls you receive from other customers?

[Probe:]

• Frequency/number of calls

• Timing

• Duration

• Types of questions/ability to respond

26. What are the most common questions/inquiries from SNAP EBT shoppers at pick-up/delivery?

[Probe:]

• Questions about eligible items

• Questions about substitutions; missing, spoiled, or damaged items

• Problems with returns, refunds, or incorrect balances

27. How do these questions/inquiries from SNAP EBT shoppers differ from those you receive from other customers?

Security/Integrity

28. What concerns do you have about transaction security?

29. Have you identified or flagged any fraudulent transactions during the pilot? 

[Probe:]

• For what reasons were transactions flagged (e.g., card sharing, identity theft, ineligible purchases)?

• What was the outcome of flagged transactions?

• How were the fraudulent transactions completed?

• What adjustments were made as a result of the fraudulent transactions you identified? (e.g., SNAP EBT customer asking the retailer to refund their money because they did not make or receive an order). How did the adjustments differ in value and/or frequency from credit/debit chargebacks?

• What would help prevent these issues in future transactions?

30. What issues or error messages do you encounter with EBT cards? For example, insufficient funds, bad PIN, refund rejections, etc.

[Probe:]

• In general, how does this compare to credit card or debit card transactions?

31. Do you use multi-factor authentication to verify identity (e.g., code sent via text message; identity verification questions)? What percentage of all online shoppers successfully pass the authentication process? Do you know if that differs for online EBT customers?

32. How do you ensure the privacy and security of information provided by EBT customers?

33. Do you require verification of identity at time of in-store pick-up or delivery (i.e., to match the customer to the order)? If so what kind?

[Probe:]

• SNAP EBT card or other credit or debit card

• Order number or receipt from online transaction

• Payment at the time of pick-up

• Can other household members pick up or accept delivery?

• What if the customer has no identification?

[Probe for “brick and mortar” retailers:]

• Do you do anything differently for your online EBT customers than your in-store EBT customers?

34. How do you ensure the integrity of the picking and fulfilling process for online EBT customers’ orders? (e.g., a second staff member verifies/checks the order)

[Probe:]

• Do you consider picking and fulfilling to be two distinct activities? If so, how would you define each? How do you ensure integrity for each?

35. Thinking about ensuring the integrity of your online ordering system and your ability to provide online ordering and delivery to online EBT shoppers, what are the strengths of your approach? Which of these do you consider “best practices”?

36. What do you see as the main risks and system vulnerabilities of online EBT use? How can they be mitigated?

Successes, Challenges, and Solutions

37. Thus far, what have been the greatest successes of the pilot?

[Probe:]

• What factors contributed significantly to these successes?

• What, if anything, could be done differently to improve the operations of the pilot?

38. What have been the biggest challenges of the pilot?

[Probe:]

• Have these challenges been resolved?

• If so, how? If not, why?

• What could other pilot stakeholders (e.g., FNS, First Data, State SNAP agency, EBT host processors) have done differently to meet their goals more effectively?

• What could your organization have done differently to meet your goals more effectively?

39. What do you think are the barriers to customer participation? Have you gotten any feedback from SNAP online customers? If so, how did you receive this feedback? What if any actions did you take as a result?

40. Have you seen any change in online SNAP shoppers’ preferences for delivery or pick-up? Does this affect your thoughts about continuing to offer online EBT shopping or expanding your pilot operations?

41. Do you have any additional suggestions for pilot improvement?

42. Thinking ahead, do you have any thoughts or advice about expanding SNAP online purchasing beyond the current pilot sites, into more stores, states, or geographic areas?

What would be involved for [RETAILER NAME]? What would be the level of effort?

[Interviewer will refer to Table 2 in the Appendix to guide the response to this question.]

[Probe:]

• Have you identified a business case for expansion (i.e., does it seem profitable)?

43. Do you have any advice for other retailers considering online purchasing?

Appendix

These tables will be sent to respondents via email in advance of the interview. During the interview, the evaluation team will review and clarify the responses.

Dear [Contact Name],

In preparation for our [visit/telephone conversation] on [date], please complete the below. Please try to respond as completely and accurately as possible. Where exact figures are not available, an informed estimate is fine.

These tables will help us understand the amount of time involved in planning and implementing the Online Purchasing Pilot, as well as the level of effort it would take to expand the pilot beyond the current locations.

Please return these tables to me no later than [date]. We will discuss these forms during our [visit/telephone conversation].

If you have any questions, please contact me at [email] or [phone].

Sincerely,

[NAME]

Table 1

Please describe the staff (technical and non-technical) involved in each phase of the pilot, their estimated level of effort (in hours), and the associated cost. If you cannot provide an estimate of hours, how much total time did it take to plan and design the pilot?

Please do not include staff time and costs related to participation in the evaluation (e.g., preparing and submitting data files), as these will not be ongoing costs.

Phase	Estimated Level of Effort				Did this take more time than initially budgeted? (yes/no)	Estimated Cost (Dollars)
	Technical Staff		Non-Technical Staff			Technical Staff	Non-Technical/ Administrative Staff
	Number of Staff	Hours	Number of Staff	Hours
Pilot Planning
Pilot Development
Pilot Testing
Pilot Implementation
Support for Ongoing Operations

Table 2

Please describe the staff (technical and non-technical) that would be involved in expanding SNAP online purchasing beyond the current pilot sites, into more stores, states, or geographic areas, the time it would take (in hours), and the associated cost. If you cannot provide an estimate of hours, how much total time would it take?

Phase	Estimated Level of Effort				Estimated Cost (Dollars)
	Technical Staff		Non-Technical Staff		Technical Staff	Non-Technical/ Administrative Staff
	Number of Staff	Hours	Number of Staff	Hours
Planning
Development
Testing
Implementation
Support for Ongoing Operations

Public reporting burden for this collection of information is estimated to vary from 2 to 3.5 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services, Office of Policy Support, 3101 Park Center Drive, Room 1014, Alexandria, VA 22302, ATTN: PRA (0584-xxxx*). Do not return the completed form to this address.

1Agricultural Act of 2014, Conference Report to Accompany H.R. 2642. P. 144-147.

2Agricultural Act of 2014, Conference Report to Accompany H.R. 2642. P. 144-147.

3Agricultural Act of 2014, Conference Report to Accompany H.R. 2642. P. 144-147.