OMB Smoke and CO Alarms_Statement B_Revised 10 15 2018

INFORMATION COLLECTION REQUEST (ICR):

OMB supporting statement and privacy impact assessment for:

Survey on Usage and Functionality of Smoke Alarms and Carbon Monoxide Alarms (SCOA) in US Households.

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1. Respondent Universe and Sampling Methods

The proposed survey will include a nationally representative survey of households within the United States. Eligible respondents must be 18 years of age or older and be considered one of the heads of the household. The questions asked in the survey require knowledge regarding duration of residency, the age of the house and the equipment installed within the house. This necessitates that the respondent be a person who makes major decisions within household. A head of household will be considered a person living or staying in the home in whose name the house or apartment is owned, being bought, or rented. A probabilistic multistage sampling approach will be utilized to select sample units for this survey. A probabilistic sampling method will allow a random selection of units with a calculable probability of selection of each unit in the target population.

The sampling approach is designed to account for US Census Regions as well as US metropolitan areas of different sizes (1 million or more and less than 1 million). As such, regions and size of the metropolitan areas were used to define 8 strata (4 regions x 2 metro sizes) from which primary sampling units (PSUs) were selected proportionately to the total number of households in each region by metro size. In the selection of PSUs, the number of PSUs selected from each region is proportionate to the total number of households in that region.

Considering the three-stage sampling approach, the primary sampling units are metropolitan areas, the second sampling units are US Census Tracts, and the elementary sampling units are US Households. At the first stage, we will consider US metropolitan areas as primary sampling units (PSU), and select a random sample of 24 among the 389 US metropolitan areas (https://www2.census.gov/programs-surveys/metro-micro/geographies/reference-files/2015/delineation-files/list1.xls, accessed on 12/04/2017 at 3:04PM). Then, we will consider US Census Tracts as secondary sampling units (SSU) and select a random sample of census tracts in each the 24 metropolitan areas selected at the first stage, as well as a random sample of census tracts in non-metropolitan areas in the proximity of metropolitan areas that will be selected. Lastly, we will select a random sample of households in each census tract selected at the second stage for survey fielding.

The end-objective is to ensure that our survey sample include approximately 85.3% of housing units within metropolitan areas and 14.7% outside, which is the current distribution of US housing units between metropolitan and non-metropolitan areas. The respondents’ universe is, therefore, a random sample of US households selected from a random sample of census tracts selected from a random sample of US metropolitan areas.

More precisely, for each region and metro area size, a number of PSUs proportionate to their total number of households across US metro areas. For example, the 24 PSUs will include 3 metro areas of 1 million or more population in the Midwest: from the US Census Bureau (ACS 2016) all metro areas with a population of 1 million or more in the Midwest have a total of 12,723,743 households, representing a proportion of 12.55% of the 101,395,933 households in US metro areas. Therefore, 12.55% of the 24 PSUs are selected from metro areas of 1 million or more population in the Midwest (3 PSUs).

B.1.1 Sampling Frame

The sampling frame will consist of occupied housing units within metropolitan and non-metropolitan areas. Records from the latest housing surveys from the US Census Bureau have been used to identify Metropolitan Areas and Census Tracts.

Sampling Stage	Sampling Unit	Population Size	Sample
# 1	US Metropolitan Areas	389	24
# 2	US Census Tracts	74,002	195
# 3	US Housing Units	118,860,065	1,185

The sample selection will utilize a random list of residential addresses (Address-Based Sample or ABS) within the census tracts selected in the second stage for pre-notification and in-home survey. The list of residential addresses will be purchased from the renowned list provider EurekaFacts has worked with in the past with proven quality contact information, such as Genesis ABS from Marketing System Solutions. Genesis ABS contains addresses that cover nearly 100 percent of addresses in the United States, making it the best available address sample frame of U.S. households. EurekaFacts also has extensive experience working with Marketing System Solutions, a licensed and reputable sample vendor, which provides high quality sample and contact list records. Having an address also provides a high degree of accuracy for targeted geographic samples.

Additionally, the frame identifies type of addresses, that is, P.O. Boxes, rural routes, vacant houses, seasonal homes, and apartments. This information will be used to define what is to be included or excluded for purposes of efficiency or for special treatment, such as a single address with multiple units where not all the units can be mailed to due to inadequate information. In addition, the address information can be updated with such information as telephone numbers, names, and census tract.

B.1.2 Sampling Approach

The EurekaFacts team will adopt a multistage sampling approach to select housing units for the SCOA survey. Initially, 8 strata are defined by considering the 4 Census Regions, and within each region, metropolitan areas with a population of 1 million or more and metropolitan areas with a population of less than 1 million. To ensure that housing units in metropolitan areas (about 85.3% of US households) as well as those not in metropolitan areas are included in the survey, the following steps are considered:

1. At the first stage, a random sample of 24 metropolitan areas (PSUs) will be selected among the 389 as primary sampling units, distributed proportionately to the number of occupied housing units in the stratum.

2. At the second stage, a random sample of 192 residential census tracts (SSUs) will be selected within the metropolitan areas selected at the first stage, as well as one census tract in non-metropolitan areas in the proximity of metropolitan areas will be selected. The non-metro area census tracts will be distributed proportionately to the number occupied housing units in a given PSU.

3. At the third stage, a random sample of five to twenty-five housing units will be selected within those census tracts for the in-home survey.
   
   

The end-objective is to ensure that our survey sample include approximately 85.3% of housing units within metropolitan areas and 14.7% outside. At the fourth stage, in order to compensate and substitute for cases where homeowners selected for the survey are not available for any reason (refusal, empty units, language barrier, etc.), we will also draw a reserve sample unit, representing about 5% of the sample size.

This approach makes it possible to calculate the probability of selection of every sample units at every stage, as well as to reliably calculate design effect, sampling error, etc. and infer the findings to the housing units in the US with a calculable level of precision.

B.1.3 Sample Size

CPSC has available funding for a sample size of 1,185 households. This sample size is large enough to provide accurate estimates representing the diversity of housing and household types, attitudes and state laws on smoke alarms and CO alarms across the country. This sample size is large enough to yield a margin of error of +/-2.83% at 95% level of confidence for estimates of the survey results in a simple random sample.

The margins of error provided below are only indicative and based on assumptions of a statistical power of 80% (usual default value), a confidence interval of 95% and a target population size of 116,900,000 units¹. The actual margin of error that will be provided with survey results may slightly differ from the ones shown here, as the actual multistage sampling design will provide a slightly different margin of error overall. The specific margin of error for multistage sampling will be calculated when the actual sample is drawn. The sample will be selected three months in advance of the survey fielding to ensure that the acquired contact information for the selected housing units is most up-to-date.

Table 2. Margin of Error by Sample Size

Sample Size (N)	1,185
Statistical Power	80%
Confidence Interval	95%
Total Occupied Housing Units	116,900,000
Margin of Error (under a SRS*)	2.8%

*SRS: Simple Random Sample

For a total sample size of 1,185 households, and to ensure that 85.3% of the sample come from metropolitan areas, the sample distribution will be:

• Metropolitan Areas: 24

• Census Tracts per Metropolitan Area: 2 to 40

• Housing Units per Census Tracts in Metropolitan Areas: 8 to 40

• Census Tracts in non-Metropolitan Area: 24

• Total Housing Units in non-Metropolitan Area: 174

• Total Housing Units in Metropolitan Areas: 1,011

The overall response rate from the sampled housing units is expected to be 25%.

B.1.4 Effect Size

Considering that the sample size for the survey is fixed at N = 1,185, provided is an estimated effect size for illustration only. Assuming that 88% of households in the control group have a functioning smoke detector (based on the proportion of households with one or more functional smoke detectors in the 1992 study by CPSC), the minimum detectable difference in the proportion of households with functioning smoke detectors in the control group (of size N1) vs. the test group (of size N2) is estimated using the likelihood-ratio test estimation method. The control group and test group can be any subdivision of US households into two complementary groups. For example, control group can be homeowners and test group can be renters; or households with individuals under the age of 18 vs. households without individuals under the age of 18.

Assumptions:

• Significance level: alpha = 5%

• Statistical power: 1 – beta = 80%

• Size of the first (control) group: N1

• Size of the second (test) group: N2

• Total sample size: N (= N1 + N2) = 1,185

• Incidence of functioning smoke detector in the first (control) group: 88%

Sample (N)	Size of Control Group (N1)	Size of Test Group (N2)	Incidence in Control Group	Effect Size*
1,185	100	1,085	88%	7.42%
1,185	150	1,035	88%	6.52%
1,185	200	985	88%	5.95%
1,185	250	935	88%	5.57%
1,185	300	885	88%	5.30%
1,185	350	835	88%	5.10%
1,185	400	785	88%	4.96%
1,185	450	735	88%	4.86%
1,185	500	685	88%	4.80%
1,185	550	635	88%	4.77%
1,185	592	593	88%	4.78%
1,185	600	585	88%	4.78%
1,185	650	535	88%	4.81%
* Estimated difference between control group and test group for a two-sample proportions test (likelihood-ratio test)

B.2. Procedures for the Collection of Information

The survey will use a mixed-mode, multi-stage approach to data collection. Prior to administering the survey, pre-notification letters will be sent to residents from the sample. The letter will explain the purpose of the study, who is sponsoring it, details on how their responses and PII will be kept secure, as well as explain the importance of the research effort. The pre-notification letter will provide credibility to our survey effort and encourage respondents’ cooperation. One to two weeks after respondents have received their pre-notification letter notifying them of the study, EurekaFacts will follow up the letter with a telephone call (where a phone number can be obtained) to screen the potential respondents. Depending on the status of smoke alarm installation or its type, eligible respondents will be scheduled for either an in-home survey interview or telephone survey.

In situations when a telephone number may not be available for the household, EurekaFacts will follow up their prenotification letter with one additional letter, inviting them to participate in the study. The second letter mailing will be sent to households without a telephone number, approximately 2-3 weeks after the initial mailing. Also, the letter will again communicate the objectives of the study and invite the individual to either reach out to EurekaFacts via telephone or complete an online intake form so a call center representative can contact them to participate in the screening process.

Depending on the status and type of their smoke alarm installation, respondents will be eligible to participate in either a telephone survey or in-home interview. During the telephone screening call, call center representatives will determine the status of the resident’s smoke alarms installation. Residents who have a smoke alarm, that is not connected to a central or security alarm that will notify the police or fire department, will be eligible for in-home interviews. If, however, they do not have a smoke alarm, or, if they do, but it is connected to a central or security alarm, the respondent will be eligible to participate in the telephone survey. The survey instrument will be programmed on Vovici software and will be administered via either in-home interviews using a Computer Assisted Personal Interview (CAPI) format or telephone, using a Computer Assisted Telephone Interview (CATI) format.

For those eligible to participate in the in-home interviews, a qualified two-member survey team will ask household residents questions related to smoke and CO alarms and then test their smoke and CO alarms. If any of the alarms do not work, we will offer to replace them free of charge. If, however, residents do not have a smoke alarm, they will receive a shorter version of the questionnaire over the telephone. The survey interview will take between 20 - 60 minutes, depending on if the survey is administered via the telephone (about 20 minutes) or in-home interviews (60 minutes). The individual data will not be identified to a specific person. Any data provided to the client or included in the report will be delivered in the aggregate form.

B.2.1 Statistical Methodology for Stratification and Sample Selection

The survey's sample selection, and sampling methodology is discussed earlier in Question 1. A probabilistic multistage sampling approach will be utilized to select sample units for this survey. A probabilistic sampling method will allow a random selection of units and an equal chance for inclusion in the survey. Considering that there are more than 134 million housing units in the nation, a simple random sampling, a systematic sampling, a stratified sampling, or a cluster sampling will be too costly and less likely to capture the diversity of the smoke alarms and CO alarms adoption, operationality, and local jurisdictions laws and regulations. The proposed multistage sampling approach will consider metropolitan areas as primary sampling units, residential census tracts within metropolitan areas as secondary sampling units, and housing units within those residential census tracts as the final sample units. At each stage a simple random sampling will be applied to select units. This approach makes it possible to calculate the probability of selection of every sample units at every stage. As such, we will be able to reliably calculate estimates and sampling error and infer the findings to the housings in the US with a calculable level of precision.

The American Community Surveys data from the U.S. Census Bureau was used to identify and quantify the number of U.S. Metropolitan Areas, the number of US Census Tracts overall and within each metropolitan area, and finally, the number of housing units overall and within each census tract. Those entities—Metropolitan Areas, Census Tracts and Housing Units—are considered as defined by “U.S. Census Bureau, Population Division, based on Office of Management and Budget, July 2015 delineations.” Based on the sampling units, a simple random sample will be selected from the qualified target at every stage. The sample selection will utilize a random list of residential addresses (Address-Based Sample or ABS) within the census tracts selected in the second stage for pre-notification and in-home survey. The list of residential addresses will be provided by Genesis ABS from Marketing System Solutions.

B.2.2 Estimation Procedure

Estimates will be produced using standard survey estimation procedures. Survey estimates include estimates of operability of smoke alarms and CO alarms, estimates of percentages of households as well as subgroups with installed of smoke alarms and CO alarms, estimates of the proportions of respondents demonstrating hazard awareness. EurekaFacts will consider and compare different methods of variance estimation such as Taylor series approximation or various replication (Jackknife, Balanced Repeated Replication) methods.

B.2.3 Unusual Problems Requiring Specialized Sampling Procedures

We chose a multi-stage sampling approach for its cost-effectiveness, time-effectiveness and flexibility. Vacant households are excluded from this survey. The U.S. Census Bureau estimates that 12% of housing units in the US are vacant; but we will consider the vacancy rate in the Census Tracts selected at Stage 2. The total number of housing units selected at Stage 3 will account for both the vacancy rate and the expected response rate to arrive at the desired number of surveys completed in that census tract.

B.3. Methods to Maximize Response Rates and Deal with Non-response

To maximize response rates, EurekaFacts’ approach includes multiple compounding efforts. First, a pre-notification letter will be sent to the sample of households selected for the survey explaining the purpose of the survey and informing them of a potential telephone call they may receive from EurekaFacts. The letter will also include a telephone number and a link to an online form providing potential respondents an alternative option to contact EurekaFacts and express their interest in the survey participation. In situations where a telephone number is unavailable, a follow-up letter will be sent to respondents, encouraging them to reach out to EurekaFacts themselves. One to two weeks after respondents have received their pre-notification letter notifying them of the study, EurekaFacts will follow-up the letter with a telephone call (where a phone number can be obtained) to screen the potential respondents and schedule them for an interview. The telephone follow-up protocol employed by EurekaFacts is designed to enhance response and cooperation rates, and thus data quality. Interviewers will attempt a total of up to ten callbacks to each number in the sample, if necessary, with seven taking place during weeknights and weekends.  Follow-up calls to households yielding no answers, busy signals, or answering machines are scheduled at varying times, including mornings and afternoons, depending upon the times that previous contacts were attempted. Depending on if residents have a smoke alarm installed or not, eligible respondents will be scheduled for either the in-home interview or the telephone survey.

If a respondent does not have a smoke alarm installed in their home, or if they do, but it is connected to a central smoke alarm, the call center representative will attempt to conduct the telephone survey during that phone call, or if the respondent is unable to complete it then, they representative will schedule it for another time. Prior to scheduling the in-home interview session, an email will be sent to the respondent reminding them of the date and time of the interview. The in-home survey will include a two-member team consisting of a fire inspector or firefighter from a local fire department and a survey professional. The survey team for an in-home survey will include a fire inspector or firefighter in order to reassure the respondents of the legitimacy of the team’s presence and gain cooperation, as well as ensure accurate results of alarms inspections and testing. The team will carry with them a letter printed on official letterhead with endorsements from the local fire department and CPSC, should they be needed.

In the SCOA survey, two types of non-response may occur. The first type is unit non-response, which occurs when data is not obtained for the sample unit (i.e., a respondent chooses not to participate in the survey). The second type is item non-response, which occurs when a respondent fails to answer one or more of the survey questions. For unit non-responses, EurekaFacts anticipate a response rate below 80% and will therefore conduct a non-response bias analysis. We will use households’ characteristics selected from the American Community Surveys data of the US Census Bureau to assess whether there exists a significant difference between households who responded to the survey and those who did not. If the non-response analysis reveals the necessity to weight the survey data, we plan to use a weighting scheme that makes respondents representative of their respective metropolitan area, as oppose to an overall weighting scheme.

Since the survey is administered by interviewer, item non-response has low likelihood. Nonetheless, a survey entry will be considered as complete only when 80% of the survey questions have been answered, skip patterns excluded. Based on the initial non-response analysis the appropriate approach to handling missing data will be identified and whether simple or multiple-imputation is required.

B.4. Test of Procedures or Methods to be Undertaken

The current survey content was developed by Vision 20/20 and Tridata, LLC, which specialize in fire safety and engineering, and reviewed by specialists with expertise with fire and CO safety, including staff from fire departments, Red Cross, subcontracted engineering firms, and CPSC staff. EurekaFacts reviewed instruments to ensure adherence to survey design guidelines and ensure an efficient and accurate data collection process which would minimize overall administration length and reduce the burden placed both on the respondents and the survey team. Three survey methodologists and two statisticians reviewed the survey and provided recommendations for ensuring relevance, clarity, and minimal response burden.

The full survey instrument underwent testing via cognitive interviews with 18 household residents (OMB Control Number 3041-0136). Cognitive testing was carried out to ensure that any questions that were misunderstood by respondents or that were difficult to answer would be improved prior to the survey fielding, and thus increase the overall quality of survey data and the accuracy of the study results. The respondents for the cognitive interviews were recruited using a slightly modified version of the survey screener (modified to meet the needs of recruiting for cognitive interviews). Recruitment was conducted using a mixed-mode methodology that combined posting on community websites such as Craigslist, and telephone outreach to follow-up specific leads generated by the advertising. The interview sessions lasted for approximately ninety minutes. The cognitive interviews were conducted with two groups of participants: 1) individuals who report having a smoke alarm that is not connected to a central alarm that may notify the police or fire department, and 2) individuals who do not have a smoke alarm installed, or if they do, their smoke alarm is connected to central or security alarm. Similarly, as planned during the actual survey fielding, during the cognitive pretesting the first group (with smoke alarm) was administered the full in-home version of the survey instrument and the second group (no smoke alarm or central alarm) was administered the telephone version of the survey. The cognitive interviews examined how well the questions performed and ensured that the material was clear and easy to understand among potential survey respondents. As part of this effort, the cognitive testing was designed to assess the question‐response process in terms of respondent’s comprehension, information retrieval process, judgment as to providing requested information, and perceived degree of ease or difficulty experienced in formulating accurate/correct responses to each question posed.

Overall the cognitive interviews demonstrated that the survey instrument did not pose any considerable challenges to the respondents. A majority of the survey questions were clear and easy to understand, and the response categories for multiple-choice questions were relevant. The testing identified minor misunderstanding or different ways of interpreting a few terms. Based on the pretesting results, participants’ comments and suggested improvements of the question wording were identified and implemented. By utilizing this method prior to survey fielding, this helped to increase the overall quality of the survey data and the accuracy of the study results.

Furthermore, to continue minimizing burden while improving utility, after the final instrument is approved and during full-field interviewing, EurekaFacts will conduct the survey fielding and administration procedures and pause after a sample of 50 respondents to conduct a data quality evaluation as to identify any issues with the instrument or data collection procedures. After the initial 50 respondents, we will evaluate the overall process of data collection, including: initially contacting the respondent, scheduling the interview, introduction, survey administration, evaluating the fire and smoke alarm, and concluding the interview. The purpose of the evaluation is to identify any risks to the data quality and inefficiencies in the data collection procedures to rectify them prior to fielding the full population or sample size.

After completing the initial set of survey interviews (n=50), the results will be processed and analyzed to identify any issues with regards to the survey instrument and data collection procedures. If any changes to the survey are needed due to pretesting results, CPSC will make the changes and submit a non-substantive change request to OMB. If the field testing did not identify any issues and no changes are required to the survey administration, the data collected during the initial 50 respondents will be incorporated into the overall sample.

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Consulted on statistical aspects of the design and to collect and analyze the data:

Jorge Restrepo (240) 403-1636

Bohdana Sherehiy, PhD (240) 403-1637

Djass Mbangdadji (240) 403-1640

EurekaFacts, LLC

Matthew Brookman (301) 987-2467

Arthur Lee (301) 987-2008

Consumer Product Safety Commission

Notification of Change to the Sampling Methodology for a Selection of Primary Sampling Units (PSUs) in the National Smoke and CO Alarm Survey (SCOA Survey)

It has come to our attention that some of the selected PSUs in the survey are designated disaster areas and/or have been severely affected by fires or hurricanes in the last couple years. FEMA has provided this information to the CPSC project team and EurekaFacts. Affected areas may include locations in North Carolina, Texas, and California. The contractor, EurekaFacts, has provided and developed a plan, if needed, to address possible low response rates in these areas. The CPSC staff project team has reviewed the plan with EurekaFacts and determined that this plan is acceptable in planning for possible low response rates in these areas.

The existing selection of PSUs and secondary sampling units (SSUs) would not be resampled. The counties that may have been severely impacted by natural disasters as identified by FEMA and that coincide with specific SCOA Survey Census Tracts could have response rates as low as 15%, instead of the forecasted response rate of 25% that was presented in the Supporting Statement approved by OMB. To anticipate for possible low response rates, accounting for the possibility of greater hardship to administer the survey in disaster impacted areas, EurekaFacts plans to adjust the expected response rate downward and to increase the number of sample household units in these locations. For example, if a PSU location had an original expected number of sample households of N=33, based on a 25% response rate, the sampling would be increased to N=55 households based on a 15% response rate. Using the original sampling strategy, anticipating a 25% response rate, it is expected that for every one (1) complete four (4) households will need to be contacted, while under the more rigorous strategy, anticipating a 15% response rate in the FEMA impacted areas, it is expected that for every one (1) complete 6.67 households will need to be contacted.

This plan minimizes impact on the already prepared and implemented survey logistics and planning and maintains the current project schedule. If additional locations are identified as potentially having low response rates, a similar plan to increase the share of sampled housing units would be used to achieve the total number of homes surveyed as defined by the survey’s overall OMB approved methodology.

1. U.S. Census, Quick Facts: Population Estimates. (2016), https://www.census.gov/quickfacts/table/PST045216/00

8