PART B_HH Pulse Survey_to OMB_041720_rev7PM 1

OMB Information Collection Request

Supporting Statement B

U.S. Department of Commerce

U.S. Census Bureau

Household Pulse Survey during the COVID-19 Epidemic

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

As noted in Part A, the Household Pulse Survey is in experimental phase. Fielding this survey during the COVID-19 epidemic is designed as a proof of concept. As such, the methodology detailed here may adapt over time. As the methodology adapts, we will submit non-substantive changes to this information collection request to OMB. In addition, we will provide all information about sampling, weighting, post-survey processing, and cognitive testing results on our website to maximize transparency for the public. This transparency is especially important given the predicted value to decision makers of having weekly state-by-state estimates of the health, social, and economic characteristics of their package. In addition to Attachment E in Part A which provides further detail on the Census Bureau’s rationale and approach, Part B, Attachment B herein sets forth a weighting methodology for this survey effort.

1. Universe and Respondent Selection

The universe for the Household Pulse Survey is the approximately 140,000,000 housing units in the United States. For operational efficiency, only housing units linked to one or more email addresses or cell phone numbers will be eligible for the sample. One respondent from each housing unit is selected to respond for him or herself. The initial sample for week 1 is approximately 2,159,000 housing units, with an additional approximately 1,100,000 housing units each additional week of data collection.  We expect to successfully receive 108,000 responses each week.  The total sample size over the 12 week period will be approximately 13.8 million housing units. This represents a 5 percent response rate.

2. Procedures for Collecting Information

Housing units linked to one or more email addresses or cell phone numbers are eligible for the sample. These housing units are stratified by state and the top 15 metropolitan areas. The sample size was determined to detect a 2 percent weekly change in key estimates at the national, state, or metropolitan area level. The sample was then allocated to states and metropolitan areas proportionally, using the number of housing units as the measure of size. Samples cases will be matched to the Census Bureau’s Contact Frame to assign email addresses. (See Attachment A for information on the Contact Frame.)

One of the main requirements of the survey is to track changes over time. To reduce respondent burden, the survey will have overlapping weekly panels that will be interviewed for three consecutive weeks and then rotate out of the sample, to be replaced by a new three-week panel. Each panel is representative of the target universe. Because response rates are expected to be low, sample will be added each week to existing panels based on their response patterns to ensure that they continue to be representative of the population. (See Attachment B for the Covid-19 Household Pulse Survey questionnaire.)

The Census Bureau will conduct this information collection online using Qualtrics as the data collection platform. Qualtrics currently is used at the Census Bureau for research and development surveys and provides the necessary agility to deploy the Household Pulse Survey quickly and securely. It operates in the Gov Cloud, is FedRAMP authorized at the moderate level, and has an Authority to Operate from the Census Bureau to collect personally identifiable and Title-protected data. The Census Bureau will utilize longitudinal data collection methodology to limit the need to re-ask questionnaire content once baseline information has been collected for continuing cases.

Weekly survey estimates will be produced by weighting the results to the estimate of the occupied number of housing units from the American Community Survey at the geographic levels of the nation, state, and metropolitan area. We expect to refine our weighting processes over time to make better use of available information and to reduce nonresponse bias in the estimates. (See Attachment C for details on the weighting plan.)

3. Methods to Maximize Response

The survey is designed to meet the goal of accurate and timely weekly estimates. It will be conducted by an internet questionnaire, with links sent by email and possibly text message. Multiple email addresses and cell phone numbers will be used to increase response for nonrespondents. These modes are expected to yield response rates much lower than traditional surveys. The benefits are implementation efficiency, cost, and timeliness of responses.

Standard errors will be large but will be available for data users to understand the quality of the estimates. Nonresponse bias is likely to be an issue, but measures such as the demographic distribution of the survey respondents compared to benchmarks will be produced for data users to consider in their analyses. There are no other known data sources that can provide comprehensive, relevant, and timely information. At the very least, this survey will provide non-anecdotal evidence about the effects of the pandemic on the population even if the quality is lower than would be traditionally acceptable.

4. Testing of Procedures

The Center for Behavioral Science Methods completed a thorough expert review that consisted of nine independent experts reviewing the questionnaire independently, followed by a consensus meeting to discuss comments and come up with recommendations. The recommendations were then passed to the Demographic Directorate, who accepted most of the expert recommendations and responded with subject-matter justifications for those recommendations that were not accepted. In the end, the questionnaire comprised a consensus of the expert survey methodologist recommendations and the subject matter needs.

Given the rapid response nature of this effort and in keeping with the agile approach in which the Household Pulse Survey is being deployed, the Census Bureau will seed the first release of sample with cognitive experts from across the statistical community and solicit feedback from them for ways the survey instrument can be improved. Additionally, we will release the survey to Census Bureau field staff working across the country, leveraging their experience as professional interviewers for feedback on the way in which respondents living in diverse communities may understand and respond to the questionnaire items. Feedback from the expert reviewers and debriefings with the field staff will be consolidated to yield recommendations for question improvements. Changes will be made and deployed in week 2 (and week 3 as necessary).

Questions identified as priorities for testing include those that were developed specifically to address the Covid-19 epidemic; items developed for potential future deployment to understand households’ decision-making in light of receiving economic stimulus payments; and questions deemed to be overly complex from a cognitive standpoint, likely as a consequence of asking agencies contributing content to keep the number of items to a minimum to reduce burden (e.g., questions on food security, questions on educational disruption).

The Census Bureau will release these data under the auspices of its Experimental Statistical Product Series. Information on the Series is available at https://www.census.gov/data/experimental-data-products.html#.

5. Contacts for Statistical Aspects and Data Collection

Statistical Design:

Anthony Tersine

Demographic Statistical Methods Division

Demographic Programs

Anthony.g.tersine@census.gov

Data Collection/Survey Design:

Jason Fields

Social Economic and Housing Statistics Division

Associate Director Demographic Programs

jason.m.fields@census.gov

Jennifer Hunter Childs

Center for Behavioral Science Methods

Associate Director Research and Methodology

jennifer.hunter.childs@census.gov

Statistical Analysis:

David Waddington

Social Economic and Housing Statistics Division

Associate Director Demographic Programs

david.g.waddington@census.gov

Attachments

1. Census Bureau Contact Frame

2. Household Pulse Survey: Weighting Approach

4