Email to OMB Staff: Bavier, Malanowski, Harris-Kojetin

To: OMB/OIRA

From: Mark Nord, Economic Research Service, USDA

Date: September 18, 2007

This email provides information requested by OMB/OIRA on September 5, 2006, in the Terms of Clearance for the Current Population Survey Food Security Supplement.

OMB/OIRA requested the following information by September 30, 2007:

1. A detailed description of the specific characteristics of food security and its measurement that make item response theory the preferred methodological and statistical basis for analyzing the data.

2. A description of both the long and short range plans for further modifications to the food security battery.

3. A description of plans for development of a hunger measure.

4. A status report regarding whether alcoholic beverages have been excluded from questions that ask about food expenditures.

1. A detailed description of the specific characteristics of food security and its measurement that make item response theory the preferred methodological and statistical basis for analyzing the data.

The conceptual definitions that underlie the operational definitions and methods embodied in the USDA measures of food security and food insecurity are:

Food Security: Access by all people at all times to enough food for an active, healthy life. Food security includes at a minimum: (1) the ready availability of nutritionally adequate and safe foods, and (2) an assured ability to acquire acceptable foods in socially acceptable ways.

Food Insecurity: Limited or uncertain availability of nutritionally adequate and safe foods or limited or uncertain ability to acquire acceptable foods in socially acceptable ways.

Several characteristics of the food security-insecurity construct suggest use of a multiple-indicator approach to measure it and the use of item response theory (IRT)-based statistical models to assess the suitability of items and sets of items for this purpose.

1-1. Food security is a research concept, not a topic of common conversation. As such, there is no commonly used, consistently understood language available to elicit information about the condition in a single question in a survey context.

1-2. Food security is not inherently categorical, but rather extends across a continuum from high food security to very low food security. Specific behaviors or experiences are characteristic of different levels of food insecurity. Information on several of these behaviors and experiences provides a more adequate indication of a households location on the continuum than can be provided by any single indicator. Measurement of food insecurity at various levels of severity is important because different adverse outcomes of food insecurity may result from different levels of severity.

1-3. Food security includes a temporal dimension (“Access … at all times….”) Assessing the food security of a household, therefore, implies either observing conditions across some extended period of time or relying on recall of a respondent to report conditions over a period of time.

1-4. For practical purposes of measurement, food security/insecurity cannot be observed directly. Food insecurity involves both a set of behaviors and experiences and the causal process of decisions contrained by a lack of adequate resources for food. Food intake can, in principle, be observed or reported, but if intake is inadequate for one or more household members, it is not generally possible to tell by observation why the intake was inadequate. The allocation of available resources for food and other basic needs involves a decision-making process of the individual (or individuals) responsible for the household’s food supply that is not accessible to direct observation. (Furthermore, in practice, it may be prohibitively expensive to observe or collect data on food intake over a sufficiently long time and with sufficient precision to meaningfully assess adequacy of the diet for active, healthy living. It would be particularly difficult to identify less severe levels of food insecurity where the impacts of constrained resources on food intake affect primarily diet quality rather than macro-nutrient sufficiency. The diets of most Americans, even of those with very adequate economic resources for food, are not very good. Thus, a poor diet observed in a food intake assessment may or may not indicate food insecurity.)

1-5. Reported behaviors and experiences that indicate food insecurity are related probabilistically, rather than deterministically, to food insecurity, for at least three reasons. First, there is reporting variability because different people may understand the questions to refer to somewhat different objective conditions. Second, there is reporting variability because, as in almost all survey questions, respondents’ ability to retrieve information from memory and classify it into an available response category is not perfect. Third, different individuals and households manage their food situations differently under conditions of constrained resources for food. Thus, combining information from several questions provides a more reliable measure of food insecurity than relying on any single question.

Given these characteristics, a multiple-indicator measure of food security/food insecurity is suggested. A strength of an IRT-based measure as opposed to an ad-hoc multiple-item index is that IRT models provide the basis of statistical tests to assess whether each item is sufficiently strongly and consistently related to food insecurity to justify including it in a measure. The IRT-based assessment also indicates how items should be combined (e.g., weighted versus unweighted index) to measure the underlying condition. The IRT model assumes a specific statistical relationship between each item and the underlying construct. These assumptions can then be tested empirically by examining the relationships among the observed items. In the development of the food security measure, this process was used to assess an initial set of candidate items and select only those that evidenced consistency with the assumptions of the selected IRT model.

2. A description of both the long and short range plans for further modifications to the food security battery.

Several changes were introduced in the December 2006 survey in accordance with recommendations of the CNSTAT panel. These were changes that did not require pre-testing and that the Census Bureau was able to make with relatively short lead time.

2-1. Alternative fills (I/We) were specified for SS1, depending on how many individuals are in the household.

2-2. Alternative fills (you/your household) were specified for SS3 and SS4, depending on how many individuals are in the household.

2-3. Alternative fills (you/your household) were specified for SS5, SS6, and SH1, depending on how many adults are in the household.

2-4. Statement of the resource constraint was standardized to read, “because there wasn’t enough money for food” in SHMF3, SH4, SHMF4, and SH5.

Planned for December 2007:

2-5. The items in the main food security series will be re-ordered to group the child-referenced items together following the adult items. This was recommended by the CNSTAT panel to reduce respondent cognitive burden. Specifically, items SS5 through SHM1 will be moved to follow SSHMF1.

2-6. Two alternative wordings will be tested for SS4 (“We couldn’t afford to eat balanced meals.” Was that OFTEN, SOMETIMES or NEVER true for (you/ your household) in the last 12 months?). The alternatives are:

“(I/we) couldn't afford to eat nutritious meals.” Was that OFTEN, SOMETIMES or NEVER true for (you/ your household) in the last 12 months?

“(I/we) couldn't afford to eat the quality and variety of foods that (I/we) should.” Was that OFTEN, SOMETIMES or NEVER true for (you/ your household) in the last 12 months?

The “balanced meals” question has been identified in several studies as less consistently associated with food insecurity (as measured by the current scale) than the other items. The CNSTAT panel also highlighted this item for further assessment. Both of the alternative questions have been tested previously either in the Food Security Supplement or in other surveys. The severity levels of both items were found to be similar to that of the “balanced meals” item, and both were more consistently associated with food insecurity (as measured by the infit and outfit statistics) than is the “balanced meals” item. Each of the alternatives will be tested in two rotations (¼ of the sample) of the December 2007 Food Security Supplement. The standard “balanced meals” version will be retained in the other half of the sample. A final decision regarding wording of this question will be made based on empirical evidence of the performance of the three versions, specifically the proportion of missing responses, the item infit statistic, and the item outfit statistic. If the outcome is not definitive, the test will be continued in December 2008.

Longer range, we plan, tentatively, to split question SH2 (about adults cutting the size of meals or skipping meals) into two questions, one about cutting the size of meals and the other about skipping meals. This change could substantially affect prevalence estimates (unlike the changes described above, which are expected to improve precision incrementally, but not to substantially affect prevalence estimates). We plan to wait until decisions are made on several other technical recommendations of the CNSTAT panel (in chapter 5 of the CNSTAT report) so that all changes that may affect estimated prevalence rates are introduced at one time with a suitable bridge to assure long-term comparability of prevalence rates.

3. A description of plans for development of a hunger measure.

The CNSTAT expert panel recommended that USDA undertake a program to measure hunger, develop measures for individuals on the basis of a structured research program, and develop a modified or new data gathering mechanism.

USDA has given careful consideration to the panel’s recommendation in the context of its highest priority policy, research, and information needs and the resources available to meet those needs. We have concluded that:

3-1. The existing measure of food insecurity—with revised labels in response to another CNSTAT recommendation—is sufficient and appropriate for most USDA policy and program purposes. The ongoing measurement and monitoring project provides needed information on households’ economic access to adequate food, quantifies the extent of unmet need, and identifies the most food-needy subpopulations.

3-2. Existing, validated measures of other health, nutrition, and child development conditions that are potential outcomes of food insecurity are collected in several ongoing surveys. Data from those surveys provide a base for identifying and quantifying a broad range of adverse outcomes of food insecurity. It would be of interest to also quantify the extent of hunger and its relationship to food insecurity. However, it appears likely that such information would add only incrementally to the range of adverse outcomes already known to be associated with food insecurity while entailing the considerable cost of developing and fielding a survey module.

3-3. Substantial additional resources (both staff and funding) would be needed to develop and implement a measure of hunger while maintaining investment in other required commitments and compelling priorities.

Therefore, full implementation of this CNSTAT recommendation is not feasible for the foreseeable future. USDA plans, however, to encourage interested non-profit and academic researchers to begin relevant background research and can serve as a central point for information interchange among those researchers. (For example, with ERS encouragement and collaboration, a recent non-profit survey of likely voters included a series of questions on what conditions respondents thought should be described as “hunger” in government reports.)

4. A status report regarding whether alcoholic beverages have been excluded from questions that ask about food expenditures.

Three changes were introduced in the December 2006 questionnaire to specifically exclude spending on alcoholic beverages from reported food expenditures.

4-1. In Question S3, which follows the question about how much the household spent at supermarkets and grocer stores in the last week, “alcohol” was added to the list of nonfood items. The question now reads, “How much of the (fill with S2O) was for non-food items, such as pet food, paper products, alcohol, detergents, or cleaning supplies?”

4-2. The same change was made to S5, which follows the question about how much the household spent (for food) at meat markets, produce stands, bakeries, warehouse clubs, and convenience stores in the last week.

4-3. The phrase, “not including alcohol” was added to S6. The question now reads, “How much did (you/your household) spend for food at restaurants, fast food places, cafeterias, and vending machinges last week, not including alcohol purchases?”