Hip_omb_ssa 11-5-24

HIP_OMB_SSA 11-5-24.doc

Healthy Incentives Pilot Evaluation

OMB: 0584-0561

Document [doc]
Download: doc | pdf

SUPPORTING STATEMENT FOR


Healthy Incentives Pilot Evaluation



Kelly Kinnison

Office of Research and Analysis

Food and Nutrition Service

US Department of Agriculture

3101 Park Center Drive

Alexandria, VA 22302

Phone: 703-305-2124

Fax: 703-305-2576

E-mail: Kelly.Kinnison@fns.usda.gov

Table of Contents

Chapter Page


PART A. JUSTIFICATION 1


A.1. Explanation of Circumstances That Make Collection of Data Necessary 1

A.2. How the Information Will be Used, by Whom, and for What Purposes 4

A.3. Uses of Improved Information Technology to Burden Reduction 9

A.4. Efforts to Identify and Avoid Duplication 11

A.5. Efforts to Minimize Burden on Small Businesses or Other Entities 11

A.6. Consequence of Less Frequent Data Collection 12

A.7. Special Circumstances Requiring Collection of Information in a Manner Inconsistent With Section 1320.5(D)(2) of the Code of Federal Regulations 13

A.8. Federal Register Comments and Efforts to Consult With Persons Outside the Agency 14

A.9. Payments to Respondents 15

A.10. Assurance of Confidentiality 16

A.11. Questions of a Sensitive Nature 18

A.12. Estimates of Respondent Burden 20

A.13. Estimates of Other Annual Costs to Respondents 20

A.14. Estimates of Annualized Government Costs 21

A.15. Changes in Hour Burden 21

A.16. Time Schedule, Publication, and Analysis Plans 22

A.17. Display of Expiration Date for OMB Approval 24

A.18. Exceptions to Certification Statement 24



PART B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 25

B.1. Respondent Universe and Sampling Methods 25

B.1.1. Respondent Universe 25

B.1.2. Sampling Methods 26

B.1.3. Response Rates and Non-Response Analysis 36


B.2. Procedures for the Collection of Information 39


B.2.1. Statistical Methodology for Stratification and Sample Selection 39

B.2.2. Estimation Procedures 39

B.2.3. Degree of Accuracy Needed for the purpose described in the justification 40

B.2.4. Unusual Problems Requiring Specialized Sampling Procedures 45

B.2.5. Any use of Periodic Data Collection Cycles to Reduce Burden 45

B.3. Methods to Maximize the Response Rates and to Deal With Nonresponse 46

B.4. Test of Procedures or Methods to be Undertaken 49

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 51


Appendixes


A1 Round 1 Advance Letter to Study Participant Survey: English


A2 Round 1 Advance Letter to Parents of Minor Study Participants: English


A3 Round 2 Advance Letter to Study Participant Survey: English


A4 Round 2 Advance Letter to Parents of Minor Study Participants: English


A5 Round 3 Advance Letter to Study Participant Survey: English


A6 Round 3 Advance Letter to Parents of Minor Study Participants: English


A7 Round 1 Advance Letter to Study Participant Survey: Spanish


A8 Round 1 Advance Letter to Parents of Minor Study Participants: Spanish


A9 Round 2 Advance Letter to Study Participant Survey: Spanish


A10 Round 2 Advance Letter to Parents of Minor Study Participants: Spanish


A11 Round 3 Advance Letter to Study Participant Survey: Spanish


A12 Round 3 Advance Letter to Parents of Minor Study Participants: Spanish


B1 Round 1 Study Participant and Primary Shopper Survey: English


B2 Round 2 Study Participant and Primary Shopper Survey: English


B3 Round 3 Study Participant and Primary Shopper Survey: English


B4 Round 1 Study Participant and Primary Shopper Survey: Spanish


B5 Round 2 Study Participant and Primary Shopper Survey: Spanish


B6 Round 3 Study Participant and Primary Shopper Survey: Spanish


B7 In Person Script: English


B8 Voice Mail script: English


B9 Refusal Avoidance and Conversion: English


C Automated Multiple Pass Method (AMPM) Screenshots


D1 Focus Group Participant Telephone Recruitment Script: English


D2 Focus Group Participant Consent Form: English


D3 Focus Group Protocol: English


D4 Focus Group Data Sheet: English


D5 Focus Group Participant Incentive Receipt Form: English


E1 Round 1 Participating Independent Store Survey


E2 Round 2 Participating Independent Store Survey


E3 Round 1 Participating Chain Store Survey


E4 Round 2 Participating Chain Store Survey


E5 Non-Participating Store Survey


E6 Withdrawn Store Survey


E7 Newly Participating Store Survey


E8 Round 1 and Round 2 Store Observation Form


E9 Follow-up Reminder Telephone Script


E10 Retailer Telephone Notification Recruiting Script

F1 Round 1 Massachusetts Executive Interview Guide


F2 Round 2 Massachusetts Executive Interview Guide


F3 Round 3 Massachusetts Executive Interview Guide


F4 Round 1 DTA Staff HIP Interview Guide


F5 Round 2 DTA Staff HIP Interview Guide


F6 Round 3 DTA Staff HIP Interview Guide


F7 Round 1 DTA Technical Team HIP Interview Guide


F8 Round 2 DTA Technical Team HIP Interview Guide


F9 Round 3 DTA Technical Team HIP Interview Guide


F10 Round 1 Local Office Interview Guide


F11 Round 2 Local Office Interview Guide


F12 Round 3 Local Office Interview Guide


F13 Round 1 EBT Contractor (ACS) Interview Guide


F14 Round 2 EBT Contractor (ACS) Interview Guide


F15 Round 3 EBT Contractor (ACS) Interview Guide


F16 Round 1 Integrated Retailers’ Interview Guide


F17 Round 2 Integrated Retailers’ Interview Guide


F18 Round 3 Integrated Retailers’ Interview Guide


F19 Round 1 Third Party Processors (TPPs) Interview Guide


F20 Round 2 Third Party Processors (TPPs) Interview Guide


F21 Round 3 Third Party Processors (TPPs) Interview Guide


F22 Round 1 Community Based Organization Interview Guide


F23 Round 2 Community Based Organization Interview Guide


F24 Round 3 Community Based Organization Interview Guide


G1 Federal Register Notice Comment


G2 National Agricultural Statistics Service Comments


H Technical Work Group (TWG): Evaluation Design and Study Plan


I IRB Approval


J Analysis Plan


PART A. JUSTIFICATION

A.1. Explain the Circumstances that Make the Collection of Information Necessary.

Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

This is a new information collection. The Food, Conservation, and Energy Act of 2008, also known as the 2008 Farm Bill (Public Law 110-246), Section 4141(K)(3)(E) authorized funds for pilot projects to evaluate health and nutrition promotion in SNAP to determine if nutrition education and incentives provided to SNAP recipients at the point-of-sale increase the consumption of fruits, vegetables, or other healthful foods. The legislation also provided for an evaluation of the funded pilot project. On the basis of this legislative authority, the U.S. Department of Agriculture (USDA) Food and Nutrition Service (FNS) designed the Healthy Incentives Pilot (HIP) and its evaluation. This OMB package requests clearance for HIP evaluation data collection activities planned for 2011-2012. This section contains an overview of the pilot and discusses the objectives of the evaluation.


Background

Reducing the prevalence of obesity is a key national health objective. In response to the increase in obesity and chronic disease in the United States, the nation’s scientific authorities and policy leaders emphasize the goal of increasing fruit and vegetable intake.1,2 Fruit and vegetable intake reduces the long-term risk of obesity,3 which threatens both health and economic welfare. Fruit and vegetable intake are associated with lower rates and reduced risk of chronic diseases, including the major causes of death in the United States: heart disease and cancer.

Most U.S. adults fail to meet the fruit and vegetable intake goals as specified in the Dietary Guidelines for Americans.2 Only 28.4% of adults meet the fruit guidelines, and only 32.5% of adults meet the vegetable guidelines,4 and these intake patterns have not improved over time. Fruit and vegetable intake shortfalls are largest for low-income Americans and participants in the Supplemental Nutrition Assistance Program (SNAP), formerly known as the Food Stamp Program.

To address this problem, FNS designed the Healthy Incentives Pilot (HIP) which will provide financial incentives to SNAP households to encourage their purchase and consumption of ‘targeted’ fruits and vegetables. The targeted fruits and vegetables are the same foods that are eligible under the national definition for WIC fruit and vegetable cash value vouchers. These foods include fresh, canned, frozen, and dried fruits and vegetables without added salt, sugars, fats, or oils. Fruit juices, mature legumes, and white potatoes are excluded, but yams and sweet potatoes are included. The intention of this incentive is to increase fruit and vegetable consumption both through the standard economic pathway of lower prices and through the implicit message of the importance of fruit and vegetable consumption that is sent by the incentive itself.5,6,7

The pilot will be operated in Hampden County, MA, by the Massachusetts Department of Transitional Assistance (DTA). FNS selected the DTA competitively based on its comprehensive pilot proposal that included very thorough and strong design, implementation, staffing and management plans. Hampden County includes a mix of 27 urban, rural, and suburban cities with approximately 53,000 SNAP households. The majority of recipients are concentrated in the areas of Springfield, Holyoke, and Chicopee. Massachusetts will operate the pilot for 14 months, beginning in late 2011 and concluding in early 2013.


Study Objectives

Congress mandated that a rigorous evaluation (using random assignment or other methods that are capable of producing scientifically valid estimates) be conducted to assess the impact of HIP on participants’ intake of fruits and vegetables. This evaluation is designed to answer the following research questions:

  • What is the impact of HIP on SNAP participants’ consumption of fruits and vegetables, other healthful foods, discretionary calories, and total calories?

  • What are the factors that influence how HIP impacts participants? Do the effects vary by household demographics, the household food environment, dietary knowledge, attitudes and beliefs, or household shopping patterns? How do participants perceive HIP and did participation affect their intention to purchase and consume fruits and vegetables?

  • What processes are involved in implementing and operating HIP? How are retailers, participants, and other stakeholders recruited? What are the challenges and lessons for future implementation?

  • What is the impact of HIP on the Pilot site and its partners? How does HIP affect workload and operations in the SNAP offices? Does HIP affect business for participating and non-participating retailers?

  • What are the costs associated with HIP implementation – i.e. initial startup, ongoing administration, and the costs of the incentives? What would it cost to implement HIP nationwide?


A.2. Indicate How, by Whom, How Frequently, and for What Purpose the Information is to be Used.

Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

This section of the supporting statement provides an overview of the HIP as well as the research design and data collection efforts planned to meet the overall objectives of the HIP evaluation.


Overview of HIP and the Evaluation Design

7,500 SNAP households within Hampden County, MA will be randomly assigned to participate in the pilot. HIP participants will earn an incentive of 30 cents for every dollar spent on targeted fruits and vegetables (TFVs). This incentive will be added to the household’s SNAP benefit account and can be used just like other SNAP benefits. The incentive payment is capped at $60 per household per month, which is a level sufficiently high that few households are expected to be constrained by the cap.8

The primary objective of the evaluation relates to the impact of HIP on SNAP participants. In order to evaluate HIP’s impact, we will randomly select a sample of 5,070 SNAP participants to take part in the evaluation – this includes 2,535 participants from the universe of 7500 HIP demonstration participants and 2,535 SNAP participants from the remainder of the SNAP caseload not receiving HIP to serve as the control group9. Thus, the demonstration and control samples will both be random samples from the common population of SNAP households in Hampden County, making them appropriate for comparison as a way of measuring the demonstration’s impact without risk of selection bias.

Understanding the implementation and costs of HIP and the potential for nationwide expansion—other important objectives of the evaluation—require collecting information from HIP stakeholders, including SNAP retailers(Business-for-profit), DTA State and local officials(State/Local agencies), HIP partners (e.g. Business not-for-profit community groups), and the Electronic Benefit Transfer (EBT) vendor and third-party-processors (Business-for-profit) involved in SNAP redemption. Samples of 75 retailers participating in HIP and a similar number of retailers choosing not to participate will be selected. Approximately 30 other stakeholders will be selected to provide information on HIP implementation and operations. The rest of this section provides an overview of the data collection efforts.

SNAP Participants – sampled SNAP HIP participants and SNAP non-HIP participants will complete three rounds of a computer-assisted telephone interview (CATI) survey at three points in time based on HIP implementation timelines: (1) at baseline prior to HIP implementation/participation, (2) 3 months into HIP implementation/participation, and (3) 11 months into HIP implementation/participation. Participants will be sent an advance letter explaining the study and the importance of their participation (Appendix A1 - Appendix A6), and interviewers will contact them within the next few weeks to conduct an interview.

When possible, the questions on the participant survey have been taken from well-validated existing surveys suitable for the study population, including NHANES10, NCI EATS11, NCI Food Attitudes and Behavior12. The survey is divided into two questionnaires: the Study Participant and the Primary Food Shopper questionnaires. Round specific annotated paper versions of the English surveys are included in Appendix B1 to B3 and Spanish surveys are included in Appendix B4 to B6 respectively. The Baseline Interview will collect information on:

  • Fruit and Vegetable intake,

  • Participation in nutrition assistance programs,

  • Exposure to nutrition education,

  • Family food environment,

  • Food preferences and beliefs,

  • Barriers to consuming fruits and vegetables,

  • General shopping patterns,

  • Food expenditures,

  • Household composition,

  • Demographics, and

  • Employment status.

While demographics and participation in nutrition assistance programs will only be collected in Round 1, the other topics listed above will also be included in Rounds 2 and 3 of the interviews. In addition, Rounds 2 and 3 will collect information on experiences with HIP (HIP participants only) and additional detail on dietary intake, using a 24-hour dietary recall protocol (Appendix C). We will use USDA’s Dietary Intake Data System to process, and code the dietary data13. This system includes the AMPM, the Post Interview Processing System (PIPS), and the SurveyNet. The Respondent and Primary Shopper surveys were cognitively tested in English and Spanish.

A sample of HIP participants not involved in the survey data collection will participate in focus group discussions. We will use the focus groups to listen to and gather information from participants regarding the implementation and operation of the pilot. Focus group topics include: how participants learned about HIP and the training they received; perceived changes in food spending and consumption after HIP implementation; and use of HIP benefits, including ease of use, process at checkout, and changes in shopping patterns.

We will conduct two rounds of focus groups, with three focus groups in each round. The first round will be conducted at the same time as the Round 2 participant interviews (3 months post-implementation) and the second group will occur at the same time as the Round 3 interviews (11 months post-implementation). A copy of the discussion guide and related focus group materials are found in Appendix D1 to D5.

Retailers – The retailer survey will collect information regarding issues surrounding the implementation and operation of HIP. A stratified sample of chain retailers, independent retailers and farmers’ markets will complete two rounds of a mail survey. Respondents include both HIP-participating retailers and non-HIP-participating retailers. The first round of the survey will occur prior to HIP implementation and the second round will occur late in HIP implementation (approximately 11 months after HIP begins). Topics covered in the first round include: store characteristics, recruitment to HIP and reasons for participating or not participating, preparations for the HIP demonstration, and fruit and vegetable inventory. The second round will focus on how HIP has affected the retailer, including: problems experienced, changes in stock of fruits and vegetables, and perceived effect on sales.

Store observations in a small sample of retailers (10 observations at three points in time, roughly corresponding to the rounds of the participant survey) will supplement the retailer survey, collecting first-hand data on transaction processes, fruit and vegetable inventory, and other environmental factors. Copies of the survey and observation form are shown in Appendix E1-E8.

Stakeholders – The local and State SNAP officials, local and State partners, EBT vendors and 3rd party processors will complete three rounds of in-person interviews to provide detailed information about the development, implementation, and operation of HIP in Hampden County. The rounds will coincide with the rounds of the participant survey and will focus on the following topics:

  • HIP implementation and operations,

  • EBT system modifications for HIP,

  • Notifying and training retail participants,

  • Impact of HIP on the State and its partners,

  • State and local costs of the pilot, and

  • Feasibility of HIP expansion.

Copy of the stakeholder interview guides are shown in Appendix F1-F24.

In addition to the data collection efforts described above, we will use administrative data provided by FNS and DTA for sampling and analysis purposes. Administrative data will include:

  • SNAP Case Files—demographic, household, eligibility, recertification and issuance information for persons in households receiving SNAP benefits. The SNAP Case File will be used to identify the sampling frame and conduct random assignment. It will also serve as a source of household characteristics for analysis.

  • Administrative Costs Files—data on the HIP-related implementation and operating costs for the State and local SNAP agencies, and costs for the EBT vendor and retailers reimbursed by the State.

  • Daily EBT Transaction Files—data on SNAP issuance, HIP and non-HIP purchase transactions, HIP incentive credits, and other transactions for each EBT card issued to a SNAP household. These files will be used to examine the take-up of the HIP incentive by participants and to examine the impact of HIP on retailer redemptions.

  • Retailer Files (STARS II and other DTA/FNS provided data)—retailer characteristics to be used for sampling retailers and in analysis of the retailer survey and the transaction data.


Use of the Information

The information gathered in the data collection activities described above will be used by FNS to determine if SNAP recipients participating in HIP have higher fruit and vegetable consumption than recipients who did not receive the incentive. The data will also permit analysis of how impacts vary by recipient characteristics. The data collection is also essential for allowing FNS to determine the potential implications of a nationwide HIP-like program. There is currently no other effort that can address the research objectives of the proposed study.


A.3. Describe whether, and to what Extent, the Collection of Information Involves the Use of Automated, Electronic, Mechanical, or Other Technological Collection Techniques or Other Forms of Information Technology, (e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection).

Also, describe any consideration of using information technology to reduce burden.


Automation of Participant Data Collection. In compliance with E-Government Act 2002, to reduce burden to the respondent and improve data quality, the HIP participant survey will implement the use of computer-assisted data collection technology. The use of computer-assisted data collection technology reduces the survey completion time – automatic skip patterns that are built-in to the program ensure that respondents are asked only relevant questions, based on their response. The branching and skip patterns applied by the system will also prevent staff from mistakenly skipping sections, omitting questions, or asking the wrong questions during interviews.

Other Data Collections. Stakeholder interviews will be conducted in person as this is considered the best method for collecting data on program implementation and operations. The interviews are designed focus on the evaluation objectives and questions will be asked of the most knowledgeable respondent. The retailer survey is designed as a paper and pencil self-administered questionnaire. This methodology is deemed most appropriate for the respondent population—many small retailers have limited access to computers and limited familiarity with web-based surveys. Sample tracking for these data collections will be managed using an Access database.


24-Hour Dietary Recall Automated Multiple Pass Method (AMPM). 24-Hour Dietary Recall Automated Multiple Pass Method (AMPM).  In order to obtain valid estimates of the usual nutrient intake in a study population, participants will complete a 24-hour dietary recall interview (24HR) which utilizes the Automated Multiple Pass Method (AMPM). (The study design requires a second recall to be collected on 10 percent (300) of the sample.) The AMPM is made possible by computer-assisted data collection technology. Trained telephone interviewers at the Westat call center will submit participants’ responses via the internal software program developed by the USDA Agricultural Research Service and Westat (see http://1.usa.gov/j4jf2Z).

FNS estimates 100 percent of these responses will be electronically submitted.


A.4. Describe Efforts to Identify Duplication.

Show specifically why any similar information already available cannot be used or modified for use for the purpose described in item 2 above.

The data requirements for the evaluation have been carefully reviewed to determine whether the needed information is already available. There is no duplication of the data to be collected in this evaluation of the Healthy Incentives Pilot. HIP represents the most dramatic effort to promote fruits and vegetables ever attempted in the 35-year history of the modern SNAP program. Past research suggests that the HIP financial incentive holds promise, but the proposed information collection is required to evaluate the approach.14,15 Information required for analysis that has been determined to be accurately available elsewhere, such as participant status information previously collected, updated, and available in SNAP case files, will be obtained from the existing records and will not be duplicated in this study.


A.5. If the Collection of Information Impacts Small Businesses or Other Small Entities, Describe any Methods Used to Minimize Burden.

Information being collected has been held to the minimum required for the intended use. A sample of retailers will be asked to complete a self-administered questionnaire. FNS estimates that approximately 60 percent of retailers in Hampden County are small businesses. Therefore, across the two survey rounds, approximately 125 retailer survey respondents will be small businesses. The self-administered surveys are designed to that they can be done at a time most convenient to respondents. If respondents’ request, the survey will be completed by telephone.


A.6. Describe the Consequence to Federal Program or Policy Activities if the Collection is Not conducted or is Conducted Less Frequently, as well as any Technical or Legal Obstacles to Reducing Burden.

This is a single-time study. Respondents to the HIP participant survey will be asked some questions two or three times over the course of the data collection period in order to evaluate the impact of HIP. The first round of interviews will take place prior to the implementation of HIP; questions are designed to capture baseline information (prior to exposure to the incentive). The design involves two rounds of surveys after the pilot begins—Round 2 will occur 3 months post-implementation and Round 3 will occur approximately 11 months post-implementation. In order to evaluate the impact of the pilot on SNAP participants’ consumption of fruits and vegetables and other foods, we need information on a variety of measures (e.g. attitudes and beliefs, exposure to nutrition education, fruit and vegetable spending and intake, family food environment), both before the pilot begins and during pilot operations to measure both shorter-term and longer-term impacts.

Retailers will also be asked some questions (e.g. store characteristics, fruit and vegetable inventory) at two points in time—prior to implementation and late in the implementation period. The rationale for asking questions at two different times is to understand how these factors changed over the course of the pilot.


A.7. Explain any Special Circumstances that would Cause an Information Collection to be Conducted in a Manner:

  • Requiring respondents to report information to the agency more often than quarterly;

  • Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

  • Requiring respondents to submit more than an original and two copies of any document;

  • Requiring respondents to retain records, other than health medical, government contract, grant-in-aid, or tax records for more than three years;

  • In connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study;

  • Requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

  • That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

  • Requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.


In order to obtain valid estimates of the usual nutrient intake in a study population, the generally accepted technique is to collect a second 24-hour dietary recall (24HR) from a subsample of the same respondents.16,17 Therefore, the study design requires a second recall to be collected on 10 percent (300) of the sample.

There are no other special circumstances. The collection of information is conducted in a manner consistent with the guidelines in 5 CFR 1320.5


A.8. If Applicable, Provide a Copy and Identify the Date and Page Number of Publication in the Federal Register of the Agency’s Notice, Soliciting Comments on the Information Collection Prior to Submission to OMB.

Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping disclosure, or reporting form, and on the data elements to be recorded, disclosed, or reported.

Federal Register

FNS published a notice February 7, 2011 in the Federal Register Volume 76, Number 25, pages 6,597-6,598 and provided a 60-day period for public comments. We received one comment that was not germane to the evaluation. Appendix G1 includes the comment received for the 60 day Federal Register Notice.

Outside Consultants

The information collection has been reviewed by the National Agricultural Statistics Service of USDA (NASS) with special reference to the statistical procedures. The report prepared by NASS is presented in Appendix G2. We incorporated the comments and suggestions regarding statistical procedures into Part B of the supporting statement.

FNS consulted with Federal government experts:

  • Margaret Andrews, USDA Economic Research Service, 202-694-5441

  • Sue Krebs-Smith, National Cancer Institute, 301-496-4766

  • Alanna Moshfegh, USDA Agricultural Research Service, 301-504-5040

A Technical Work Group (TWG) has been assembled and consulted regarding project design considerations for data collection and analysis. TWG members are listed in Table A8.1. The TWG members participated in a meeting with FNS and the evaluation contractor. Appendix H presents a summary of the issues discussed and feedback received at the meeting; it also presents decisions and the rationale for how input will be used to improve the evaluation. No major changes in the evaluation were recommended.



Table A8.1. Consultants from Outside the Agency


Name

Title and Affiliation

Area of Expertise

Tom Baranowski

(713-798-6762)

Professor, Department of Pediatrics

Baylor College of Medicine

Experimental research design (community settings)

Simone French

(612-626-8594)

Professor, Division of Epidemiology and Community Health

School of Public Health

University of Minnesota

Incentives to promote healthful food choices

Joel Gittelsohn

(410-955-3927)

Professor, Department of International Health

Johns Hopkins Bloomberg School of Public Health

Qualitative methods or process evaluation

David Just

(607-255-8048)

Economist and Associate Professor, Department of Applied Economics and Management

Cornell University

Incentives to promote healthful food choices

Diane Whitmore Schanzenbach

(847-491-3884)

Associate Professor, School of Education and Social Policy, Northwestern University

Faculty Research Fellow, National Bureau of Economic Research.

Food demand analysis



A.9. Explain any Decision to Provide any Payment or Gift to Respondents, Other than Remuneration of Contractors or Grantees.

Achieving high response rates is critical to data quality. Providing an extrinsic incentive increases cooperation rates, especially in populations defined as being in poverty,18,19 and a monetary incentive even more so.20 An effective incentive can improve initial response rates, which reduce the need for follow-up and decrease survey costs.21 In addition, incentives disproportionately encourage those less interested in the research to participate, thus reducing non-response bias.22 Finally, the research suggests that individuals receiving an incentive are more co-operative in providing contact information that will allow tracking their whereabouts for successive rounds of a survey.23 Based on this research, SNAP participants in the HIP study will receive a monetary incentive after completing each round of the survey: $20 for round 1, $30 for round 2, and $40 for round 3. This differentially higher incentive scale will encourage respondents to provide contact information for subsequent rounds, and will persuade them with a “promise” of a higher incentive, to continue their participation through the three rounds of data collection. Focus group participants will receive a monetary incentive of $75.

No incentive payments are planned for HIP-participating retailers, the EBT vendor, or other stakeholders; their involvement in the HIP evaluation. Participating retailers will sign a Memorandum of Understanding with the DTA that includes agreement to participate in the evaluation. The EBT vendor and most other stakeholders receiving funding from DTA and will be required to participate in the evaluation.

Securing the cooperation of the non-participating retailers and those who dropped out of the pilot may be more challenging than for participating retailers. We will encourage non-participating retailers or those who dropped out of the pilot to cooperate by noting that if a HIP-like program was to be implemented nationwide, retailers might see an increase in sales of fruits and vegetables that earned SNAP participants an incentive. We would also appeal to their interest in improving the health and nutrition of Americans participating in SNAP. Finally, we will offer a modest $40 monetary incentive to the non-participating retailers for completing a survey.


A.10. Explain any Assurance of Confidentiality Provided to Respondents and the Basis for the Assurance in Statute, Regulation, or Agency Policy.

The individuals participating in this study will be notified that the information they provide will not be published in a form that identifies them. No identifying information will be attached to any reports. Identifying information will not be included in the public use dataset. USDA will receive a dataset that includes participant IDs so that USDA can conduct follow-up research in the future using administrative data.

Abt Associates Inc. and Westat have extensive experience in data collection efforts requiring strict procedures for maintaining the privacy, security, and integrity of data. Specific data handling and reporting procedures will be employed to maintain the privacy of survey participants and composite electronic files. These data handling and reporting procedures include requiring all project staff, both permanent and temporary, to sign a confidentiality and nondisclosure agreement (Appendix I3). In this agreement, staff pledges to maintain the privacy of all information collected from the respondents and will not disclose it to anyone other than authorized representatives of the evaluation, except as otherwise required by law.

In addition, Abt Associates Inc. and Westat have established a number of procedures to ensure the privacy and security of electronic data in their offices during the data collection and processing period. A system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports in the Federal Register on March 31, 2000, Volume 65, Number 63, and is located on pages 17251-17252 discusses the terms of protections that will be provided to respondents.


Institutional Review Board

Abt Associates maintains its own Institutional Review Board (IRB), which serves as the organization’s administrative body that conducts prospective reviews of proposed research and monitors continuing research for the purpose of safeguarding research participants’ rights and welfare. All research involving interactions or interventions with human subjects is within the purview of the Abt IRB. Abt Associates’ IRB is the local agent responsible for ensuring that the organization’s research: 1) meets only the highest ethical standards; and 2) receives fair, timely, and collegial review by an external panel. Abt Associates’ IRB currently holds a federal-wide assurance (FWA) of compliance from the U.S. Department of Health and Human Services’ Office of Human Research Protections (DHHS/OHRP). The FWA covers all federally supported or conducted research involving human subjects. All study materials and instruments for the HIP evaluation were approved by Abt’s IRB. Copies of the IRB approval letters are in Appendix I1 and I2.


A.11. Provide Additional Justification for any Questions of a Sensitive Nature, such as Sexual Behavior or Attitudes, Religious Beliefs, and Other Matters that are Commonly Considered Private.

This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

Several questions on the HIP participant survey could be considered to be sensitive to study respondents. These include participant marital status, household participation in nutrition assistance programs, household income, and household food expenditures. In addition, for some participants the enumeration of their household, which requires listing all members living in the household, might be a sensitive area. Finally, when reporting their 24-hour dietary recall, participants are asked to remember “everything they ate or drank yesterday However, because the purpose of the study is to measure the impact of HIP on SNAP households and their quality of life, and such determinations are based on household characteristics (e.g., household size, number of dependents, income,), those questions are necessary to evaluate HIP. All of these questions were cognitively tested and no respondent indicated an unwillingness to answer the question or discomfort with providing a response.


A.12. Provide Estimates of the Hour Burden of the Collection of Information. The statement should:

  • Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 14 of OMB Form 83-I.

  • Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories.



Table A12.1 shows sample sizes, estimated frequency of responses, estimated annual responses, estimated burden, and estimated annualized cost of respondent burden for each part of the data collection and for total burden.

The estimated annualized cost to respondents for the HIP participant survey and HIP participant focus groups with primary food shoppers is based on the national minimum wage of $7.25 per hour. The estimated annualized cost to respondents for the retailer survey is based on the May 2009 National Occupational Employment and Wage Estimates for the United States, available at www.bls.gov/oes/current/oes_nat.htm, for job category “First-Line Supervisors / Managers of Retail Sales Workers”, occupation code #41-1011, with a median wage of $16.78 per hour. The estimated annualized cost to respondents for the stakeholder interviews (including local and State SNAP officials, local and State partners, EBT vendors/3rd party processors) is based on the May 2009 National Occupational Employment and Wage Estimates for the United States, available at www.bls.gov/oes/current/oes_nat.htm, for job category “Management Occupations”, occupation code #11-0000, with a median wage of $42.95 per hour.


Table A12.1. Respondent Burden and Cost

Affected Public

Respondent


Number
of Respondents

Average Responses Annually Per Respondent

Total Annual Responses

Estimated Hours Per Response

Estimated Total Hours

Annualized Cost of Respondent Burden

Individual or Households

SNAP recipients*

Pretest

27**

1.000

27

0.559

15.09

$ 109

Completed

2,828

2.061

5,829

.7672

4,472.01

$ 32,422

Attempted

1,212

1.720

2,085

.0521

108.63

$ 788

SNAP recipient focus groups

Completed

60

1.000

60

1.5000

90.00

$ 653

Attempted

6

1.000

6

0.0500

0.30

$ 2

SUBTOTAL


4,133


8,007


4,686.03

$ 33,974

Business for profit

Retailers

Pretest

9

1.000

9

0.5000

4.50

$ 76

Completed

165

1.000

165

0.5758

95.01

$ 1,594

Attempted

42

1.000

42

0.0500

2.10

$ 35

EBT vendors/
3rd party processors

Completed

8

1.000

8

1.5000

12.00

$ 515

Attempted

1

1.000

1

0.0500

0.05

$ 2

SUBTOTAL


225


225


113.66

$ 2222

State and Local agencies

Local and State SNAP Officials

Completed

17

1.000

17

1.4118

24.00

$ 1,031

Attempted

2

1.000

2

0.0500

0.10

$ 4

SUBTOTAL


19


19


24.10

$ 1035

Business not for profit

Local and State Partners

Completed

5

1.000

5

1.5000

7.50

$ 322

Attempted

1

1.000

1

0.0500

0.05

$ 2

SUBTOTAL


6


6


7.55

$ 324


TOTAL


4,383


8,257


4,831.34

$ 37,555

*Figures inclusive of burden for treatment and control groups

** English and Spanish surveys were tested in two rounds; no more than 9 respondents were asked the same question


A.13. Provide Estimates of the Total Annual Cost Burden to Respondents or Record Keepers Resulting from the Collection of Information (do not include the cost of any hour burden shown in items 12 and 14).

The cost estimates should be split into two components: (a) a total capital and start-up costs component annualized over its expected useful life; and (b) a total operation and maintenance and purchase of services component.

There are no capital/start-up or ongoing operation/maintenance costs associated with this information collection.



A.14. Provide Estimates of Annualized Cost to the Federal Government.

Also, provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information.

The total annualized cost to the Federal government for all tasks associated with the HIP Evaluation, including the data collection and associated project costs, is $2,993,504 per year.  This includes evaluation contractor costs associated with: (1) developing instruments, correspondence and administrative forms; (2) developing sampling plan and sample selection; (3) developing evaluation, data collection and analysis plans; (4) systems programming and testing of the data collection software and tracking systems; (5) interviewer training; (6) data collection; (7) data cleaning and processing; (8) data tabulation and analyses; (9) report writing; and (10) overall project management. It also includes costs of the FNS Project Officer for monitoring the study.

A.15. Explain the Reasons for any Program Changes or Adjustments Reported in Items 13 or 14 of the OMB Form 83-I.

This is a new collection of information and as a result of program changes will add 4,831.34 hours to FNS inventory.


A.16. For Collections of Information whose Results are Planned to be Published, Outline Plans for Tabulation and Publication.

Time Schedule

The schedule for the study showing sample selection, beginning and ending dates of collection of information, completion of reports, and publication dates is shown on Table A16.1.


Table A16.1. Data Collection and Reporting Schedule


Activity

Schedule

Sample selection

August 2011

Round 1 field period: participants, retailers, stakeholders

September–December 2011

Round 2 field period: participants, stakeholders

February–April 2012

Round 3 field period: participants, retailers, stakeholders

August–December 2012

Implementation report

March 2012

Interim report

November 2012

Final report

September 2013

Summary report

December 2013

Annual reports to Congress

December 2010

December 2011

December 2012


Analysis Plan

The primary purpose of the study is to evaluate the implementation, operation, and impact of HIP. The main lines of analysis follow the five research objectives outlined in section A.2. Exhibit A16.2 presents an overview of the research objectives, data collection activities, and reports that this study will produce for future policy development regarding point-of-purchase financial incentives. Appendix J presents the detailed analysis plan.

Exhibit A16.2. Objectives, Principal Data Sources, and Reports


Objectives

Data Sources

Reports

1. Assess the causal impact of HIP on participant fruit and vegetable consumption and other key measures of dietary intake.

Participant survey: R1, R2

Participant focus groups: R1

EBT transactions data

Interim

Participant survey: R1, R2, R3

Participant focus groups: R1, R2

EBT transactions data

Final

2. Identify and assess factors that influence how HIP impacts participants.

Participant survey: R1, R2

Participant focus groups: R1

SNAP casefile data

EBT transactions data

Interim

Participant survey: R1, R2, R3

Participant focus groups: R1, R2

SNAP casefile data

EBT transactions data

Final

3. Describe the processes involved in implementing and operating HIP.

Retailer survey: R1

Retailer observation: R1

Stakeholders*: R1

Implementation

Retailer survey: R1, R2

Retailer observation: R1, R2, R3

Stakeholders*: R1, R2, R3

Participant survey: R2, R3

Final

4. Assess the impact on the State SNAP agency, the local SNAP Agency, and partners.

Retailer survey: R1, R2

Retailer observation: R1, R2, R3

Stakeholders*: R1, R2, R3

EBT transactions data

Final

5. Quantify, to the extent possible, the Federal, State, and local administrative and benefit costs of the pilot.

Retailer survey: R1, R2

Retailer observation: R1, R2, R3

Stakeholders*: R1, R2, R3

EBT transactions data

Final

* Includes State and local agency SNAP staff, State and local partners (including community organizations), EBT vendor/Third Party Processors


Publication of Study Results

The study’s findings will be presented in reports that will undergo peer review. Once final, FNS will make the reports available on its web site. Findings may also be published in one or more professional journals and publications intended for general or trade audiences, such as nutrition educators or food retailers.


A.17. If Seeking Approval to not Display the Expiration Date for OMB approval of the Information Collection, Explain the Reasons that Display would be Inappropriate.

All data collection instruments for the Healthy Incentives Pilot Evaluation will display the OMB approval number and expiration date.


A.18. Explain Each Exception to the Certification Statement Identified in Item 19 “Certification for Paperwork Reduction Act.”

There are no exceptions to the Certification for Paperwork Reduction Act (5 CFR 1320.9) for this study.


1U.S. Department of Health and Human Services (2000). Healthy People 2010. 2nd Edition, Washington, DC: U.S. Government Printing Office.

2U.S. Department of Agriculture and U.S. Department of Health and Human Services. Dietary Guidelines for Americans, 2010. 7th Edition, Washington, DC: U.S. Government Printing Office, December 2010.

3He, K., Hu, F.B. Colditz, G.A., Manson, J.E., Willett, W.C., and Liu. S. (2004). Changes in intake of fruits and vegetables in relation to risk of obesity and weight gain among middle-aged women. International Journal of Obesity Related Metabolic Disorders 28(12), 1569-74.

4Casagrande, S., Wang, Y, Anderson, C & Gary T. (2007). Have Americans increased their fruit and vegetable intake? The trends between 1988 and 2002. American Journal of Preventive Medicine 32(4), 257-263.

5Wilde, P.E., & Andrews, M. (2000). The Food Stamp Program in an era of welfare reform: Electronic benefits and changing sources of cash income. Journal of Consumer Affairs, 34(1), 31-46.

6Wilde, P & Ranney, C. (2000). The monthly food stamp cycle: Shopping frequency and food intake. American Journal of Agricultural Economics 82, 200-213.

7Wilde, P.E., Troy, L.M, & Rogers, B.L. (2009). Food Stamps and food spending: An Engel Function approach. American Journal of Agricultural Economics 91(2), 416-430, May 2009.

8Based on three simulations using three alternative models of consumer purchasing behavior. Findings indicated that if the statutory cap is set too low, it could limit the impact of HIP on households’ expenditures for target fruits and vegetables. (HIP participants who are not capped face a price for fruits and vegetables that is reduced by 30%, whereas participants who are capped face the original price of fruits and vegetables.) However, if the statutory cap is set too high, it could encourage fraud and cause the total incentive payments to exceed the budgeted amount of $2 million.

9 We expect 20.3% attrition from SNAP between sampling and the start of data collection, yielding 2,020 respondents on SNAP in each group (a total of 4,040 participants, as shown in Exhibit A12.1.). We will attempt to contact these 4,040 respondents to complete the baseline survey.

10 NHANES: OMB No. 0920-0237; EXP DATE 12/ 31/2011


11 NCI EATS: OMB No. 0925-0450; EXP. DATE: 07/31/2000


12 FAB: OMB #0925-0560; Exp. Date: 03/31/2009


13 Raper N, Perloff B, Ingwersen L, Steinfeldt L, Anand J.  An overview of USDA’s Dietary Intake Data System.  2004.  J Food Composition and Analysis 17: 545-555.


14Andreyeva, T., Long, M.W., & Brownell, K.D. (2010.) The impact of food prices on consumption: A systematic review of research on the price elasticity of demand for food. American Journal of Public Health 100(2), 216-22.

15Government Accountability Office. (2008). Food Stamp Program: Options for delivering financial incentives to participants for purchasing targeted foods (GAO-08-415). Washington, D.C.: GPO. Available: http://www.gao.gov/new.items/d08415.pdf..

16S.M. Nusser, A..L. Carriquiry, et al (1996). A semi-parametric transformation approach to estimating usual nutrient intake distributions, J. Am. Stat. Assoc., 91: 1440-1449.

17Institute of Medicine (IOM). Dietary Reference Intakes: Applications in dietary assessment. Washington, DC: National Academies Press; 2000.

18Singer E. (2002). The use of incentives to reduce non response in households surveys in: Groves R, Dillman D, Eltinge J, Little R (eds) Survey Non Response. New York: Wiley, pp 163-177.

19James T. (1996). Results of wave 1 incentive experiment in the 1996 survey of income and program participation. Proceedings of the Survey Research Section, American Statistical Association., 834-839.

20Groves R, Fowler F, Couper M, Lepkowski J, Singer E. (2009) in: Survey methodology. John Wiley & Sons, pp 205-206.

21Berlin M, Mohadjer L, Waksberg J, Kolstad A, Kirsch D, Rock D, Yamamoto K. (1993). An experiment in monetary incentives. Proceedings of Survey Research Methods Section of American Statistical Association, 393-8.

22Groves R, Singer E, Corning A. (2000). Leverage-saliency theory of survey participation: description and an illustration. Public Opinion Quarterly. 64(3): 299-308.

23Shettle C, Mooney G. (1999). Monetary incentives in US Government surveys. Journal of Official Statistics, 15(2): 231-250.

0

File Typeapplication/msword
File TitleHealthy Incentives Pilot Evaluation
File Modified2011-05-24
File Created2011-05-24

© 2024 OMB.report | Privacy Policy