OMB Information Collection Request
Supporting Statement B
U.S. Department of Commerce
U.S. Census Bureau
Household Pulse Survey
During the Coronavirus Pandemic
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
As noted in Part A, the Household Pulse Survey was initially fielded as a proof of concept to test the Federal statistical system’s ability to produce data in near real-time in the face of a national emergency. The survey is conducted under the auspices of the U.S. Census Bureau’s Experimental Data Products Series (see https://www.census.gov/data/experimental-data-products.html).
Planning for the survey began March 23, 2020, and with OMB emergency clearance granted on April 19, 2020 for 3 months of data collection (OMB No. 0607-1013, Exp. 7/31/2020), the Census Bureau launched the Household Pulse Survey on April 23, 2020. Data were first released on May 20, 2020, with updated releases made subsequently on a weekly basis. Since then, the data have been used widely to guide response and recovery efforts on the part of Federal agencies and state governments. Given the continuing nature of the pandemic, the Census Bureau sought, and OMB approved, an extension to data collection for the balance of the 180 days allowable for emergency clearance under 5 CFR Part 1320, Section 1320.13, Emergency Processing. The Census Bureau then put forward a regular (non-emergency) Information Collection Request (ICR) for OMB review on September 17, 2020. OMB approved this ICR on October 30, 2020 for three years (OMB No. 0607-1013; expiration October 31, 2023). Given information about the trajectory of the pandemic in early December, the Census Bureau submitted a non-substantive request to OMB on December 22, 2020 to request approval for continuing collection of the current Phase 3 instrument through January and February, 2021. To continue the survey without hiatus, the Census Bureau submitted a revision request for the Phase 3.1 questionnaire in March 2021. The Census Bureau submitted a revision request for the Phase 3.2 questionnaire in June 2021, which included the removal of questions for which utility has declined over time, and the addition of topics based on public comment previously received and in consult with other Federal agencies. The revision for Phase 3.3 included modifications to existing questions and reinstatement of previously approved items as well as a shift from bi-weekly to monthly data collection. The Phase 3.4 revision included the addition of a new question, modification of existing items, reinstatement of previously improved questions, and the removal of items with declining utility. The revisions for Phases 3.5, 3.6, and 3.7 included the addition of new questions, modification of existing items, reinstatement of previously improved questions, and the removal of items with declining utility. Phase 3.8 was approved without substantive changes. The Phase 3.9 revision included new, revised, and removed questionnaire items. This Information Collection Request (ICR) is submitted for the purposes of requesting a revision to an Existing Collection for a revised, Phase 3.10 questionnaire.
For ease of reference, we refer to the initial approval by OMB to conduct the Household Pulse Survey as “Phase 1” (April – July, 2020), and the period approved for the balance of the emergency clearance as “Phase 2” (August – October, 2020). “Phase 3” is in reference to normal clearance granted starting October 30, 2020 through October 30, 2023. Subsequent revisions to the questionnaire are referred to as “Phase 3.1,” Phase 3.2,” etc. This ICR requests approval to launch “Phase 3.10” with a revised questionnaire beginning in August 2023.
Given the rapid response nature of this survey, the Census Bureau continued to make adjustments in content and methods throughout Phase 1, documenting those through the submission of non-substantive change requests to OMB. As data collection progressed, there was increasing demand from other Federal agencies to revise or add content to the Household Pulse questionnaire to produce data in support of their own missions and response to the pandemic. Rather than change the survey on a constant basis, the Census Bureau consolidated those requests into one comprehensively revised questionnaire, which was launched as Phase 2.
The Census Bureau’s approach to Phase 3 was similar to Phase 2, including the content, contact strategies, mode, data collection cycle and sampling approach. However, we continued to monitor data needs over the course of the pandemic and its recovery to determine if revisions (including additions and deletions of items) were needed. To ensure that the data collected by the Household Pulse Survey continue to meet information needs as they evolve over the course of the pandemic and other emergent social and economic issues, Phase 3.10 includes split-panel tests of two versions of the gender identity question and the placement of the health items in the instrument. The Phase 3.10 instrument also includes the return of the original Hispanic origin, race/ethnicity, childcare arrangements, and children’s mental health behaviors items, and new questions on military participation of a spouse, shortages of critical products, difficulty accessing medical care/treatment during natural disasters, and a question asking the respondent if they will opt-in to receive optional surveys and email updates from the Bureau. There are also modifications to existing children’s vaccine booster, infant formula, reasons not working, unemployment insurance, natural disaster, and contact information items. Several questions will be removed for Phase 3.10, including questions about obtaining at home COVID tests, not working due to the pandemic or natural disaster, households affected by the infant formula shortage, type of formula used, and best contact method.
As in previous phases, we will provide all information about sampling, weighting, post-survey processing, and cognitive testing results on our website to maximize transparency for the public. This transparency is especially important given the value to decision makers of having rapidly produced monthly state-by-state estimates of the health, social, and economic characteristics of their package.
Universe and Respondent Selection
The universe for the Household Pulse Survey is the approximately 140,000,000 housing units in the United States. For operational efficiency, only housing units linked to one or more email addresses or cell phone numbers will be eligible for the sample. One respondent from each housing unit is selected to respond for him or herself. The sample for each data collection cycle is approximately 1,040,000 housing units.
In Phase 1, the Census Bureau observed response rates in the range 6-7 percent for households who were invited to participate for the first time and asked to respond in a one-week period. In Phase 2, we extended the response period to 13 days and sent additional reminders to respondents. These efforts had the effect of increasing response rates to a range of 8.1 to 10.3 percent. In Phase 3, we continued with the 13-day response period and observed a response rate of between 5.3 and 7.5 percent. In Phase 3.1, Phase 3.2, and Phase 3.3, we have observed a response rate of between 5.4 and 7.4 percent. Phase 3.4 averaged a response rate of 6.6% and the response rate of Phase 3.5 started at 6.2% and eventually dropped to 4.4%. The response rate over the most recent three waves of data collection is in the range of 3.9 - 4.7 percent. For Phase 3.6, the response rate ranged from 3.9% to 5.6%. There was an increase in response rates for Phase 3.7, with rates ranging from 6.4% to 7%. Phase 3.8 had an average response rate of 5.9% and the first two weeks of Phase 3.9 averaged a response rate of 6.2%. Based on these recent experiences, we are assuming a 6.1% percent response rate for Phase 3.10 and expect to receive approximately 64,510 responses each data collection cycle for a total of 193,530 for the entire phase. Sample from each data collection cycle will be independent from the prior cycles.
Procedures for Collecting Information
Housing units linked to one or more email addresses or cell phone numbers are eligible for the sample. These housing units are stratified by state and the top 15 metropolitan areas. For an estimate of 40 percent, the sample size was determined to produce a margin of error of 2.8 percent for most states and 3.3 percent for the 11 smallest states with eligible sample units (Alaska, Delaware, District of Columbia, Hawaii, Maine, Montana, North Dakota, Rhode Island, South Dakota, Vermont, and Wyoming) with 90 percent confidence. The sample was then allocated to states and metropolitan areas proportionally, using the number of housing units as the measure of size. Samples cases will be matched to the Census Bureau’s Contact Frame to assign email addresses and cell phone numbers. See Attachment A for information on the Contact Frame.
A sampled address may have up to five associated email addresses and up to five cell phone numbers. Starting on the first day of the data collection period, an email invitation will be sent in the morning and an SMS invitation in the afternoon to the first set of contact information for the unit. If a response is not received by that evening, on the next day an email and SMS will be sent to the second set of contact information for the unit (if it exists) and so on. This process will be repeated on weekdays. In addition to the initial invitations, each unique piece of contact information may receive up to 2 additional reminders during the collection period. Data collection closes on midnight of the second Monday in the data collection period (12 days after the initial invitation).
The Census Bureau will conduct this information collection online using Qualtrics as the data collection platform. Qualtrics is currently used at the Census Bureau for research and development surveys and provides the necessary agility to deploy the Household Pulse Survey quickly and securely. It operates in the Gov Cloud, is FedRAMP authorized at the moderate level, and has an Authority to Operate from the Census Bureau to collect personally identifiable and Title 13-protected data.
Attachment B sets forth details on the Census Bureau’s weighting plan, which will use a multi-step process to help reduce nonresponse bias in the estimates.
Methods to Maximize Response
The survey is designed to meet the goal of rapid-response, accurate, and timely estimates. It will be conducted by an internet questionnaire, with links sent by email and SMS text message. Up to five email addresses and cell phone numbers will be used for each unit to increase response for nonrespondents. One reminder will be sent to each piece of contact information if a response is not received. These modes are expected to yield response rates much lower than traditional surveys. The benefits are implementation efficiency, cost, and timeliness of responses.
Standard errors will be large but will be available for data users to understand the quality of the estimates. Nonresponse bias is likely to be an issue with this design, but measures such as the demographic distribution of the survey respondents compared to benchmarks will be produced for data users to consider in their analyses. Limitations due to collection and contact modes may exacerbate issues associated with non-response by limiting options for reducing bias. Weighted distributions of key demographic covariates are included in the detailed tables and are accompanied by companion standard error tables. These distributions of these characteristics can be compared with other federal and non-federal data sources. The microdata allows for the comparison of weighted and unweighted estimates and evaluations of gaps in response. Source and Accuracy documents include coverage rates both before and after weighting raking procedures and are released with each data cycle release. There are no other known data sources that can provide comprehensive, relevant, and timely information. At the very least, this survey will provide non-anecdotal evidence about the effects of the pandemic on the population even if the quality is lower than would be traditionally acceptable.
Source and Accuracy statements and other technical documentation is available at https://www.census.gov/programs-surveys/household-pulse-survey/technical-documentation.html.
Testing of Procedures
Prior to Phase 1, the Center for Behavioral Science Methods completed a thorough expert review that consisted of nine independent experts reviewing the questionnaire independently, followed by a consensus meeting to discuss comments and come up with recommendations. The recommendations were then passed to the Demographic Directorate, who accepted most of the expert recommendations and responded with subject-matter justifications for those recommendations that were not accepted. In the end, the questionnaire comprised a consensus of the expert survey methodologist recommendations and the subject matter needs. A similar process was followed prior to subsequent phases as the Census Bureau considered questionnaire revisions to keep up with evolving data needs.
We will take a similar course for Phase 3.10: feedback from Federal agency partners, data users and other stakeholders on diminishing, changing or emerging data needs will inform changes to the questionnaire.
Phase 3.10 will include two split-panel tests; one of which will test two versions of the gender identity question with slightly different wording and different response categories; and one which will test the placement of the block of health items in the instrument – after the demographic items early in the instrument and at the end of the instrument. These two split-panel tests will be crossed into four unique treatment groups.
In Phase 3.9, the Census Bureau conducted an experiment to test two methods of operationalizing the Initial Proposals for Revising the Federal Race and Ethnicity Standards | OMB | The White House in a self-administered web format. One version of the combined race and ethnicity question presented the minimum categories, with the detailed categories unfolding on the screen after a minimum category is selected. The second version presented the minimum categories on one screen and included a follow-up screen with the detailed categories of the selected minimum categories. Analysis compared distributions between panels, considering both minimum and detailed category selection and also considered the distributions of these experimental panels as compared with data collected using the original race and ethnicity questions in prior phases of the HPS.
Given the rapid response nature of this effort and in keeping with the agile approach in which the Household Pulse Survey is being deployed, the Census Bureau, in partnership with BLS and NCHS, concurrently launched a web probing study with participants who had opted in to participate in a research study with the Census Bureau via email. These sampled participants were sent email invitations to the survey with embedded probes. Two rounds of web probing were conducted in Phase 1, and an additional round in Phase 2, Phase 3, and Phase 3.1. The Household Pulse Survey Team used results from these studies to refine content and cut questions to stay within a 20-minute limit for the survey.
The Census Bureau will release these data under the auspices of its Experimental Statistical Product Series. Information on the Series is available at https://www.census.gov/data/experimental-data-products.html#.
Contacts for Statistical Aspects and Data Collection
Statistical Design:
Anthony Tersine
Demographic Statistical Methods Division
Demographic Programs Directorate
anthony.g.tersine.jr@census.gov
Data Collection/Survey Design:
Jason Fields
Social Economic and Housing Statistics Division
Demographic Programs Directorate
jason.m.fields@census.gov
Jennifer Hunter Childs
Center for Behavioral Science Methods
Research and Methodology Directorate
jennifer.hunter.childs@census.gov
Statistical Analysis:
David Waddington
Social Economic and Housing Statistics Division
Demographic Programs Directorate
david.g.waddington@census.gov
Attachments
Census Bureau Contact Frame
Household Pulse Survey: Weighting Plan
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Dumas, Sheleen (Federal) |
| File Modified | 0000-00-00 |
| File Created | 2023-10-06 |