Supporting Statement Part B 1660-PROGNEWX

1/21/2021

Supporting Statement for

Paperwork Reduction Act Submissions

OMB Control Number: 1660 – NW103

Title: Federal Emergency Management Agency Programs Customer Satisfaction Surveys

Form Number(s):

FEMA Form 519-0-44 Preparedness –Phone

FEMA Form 519-0-45 Preparedness -Electronic

FEMA Form 519-0-46 Transitional Sheltering Assistance (TSA)–Phone

FEMA Form 519-0-47 Transitional Sheltering Assistance (TSA)-Electronic

FEMA Form 519-0-48 Temporary Housing Units (THU)–Phone

FEMA Form 519-0-49 Temporary Housing Units (THU) -Electronic

FEMA Form 519-0-50 Shelter and Temporary Essential Power (STEP) -Phone

FEMA Form 519-0-51 Shelter and Temporary Essential Power (STEP)-Electronic

To streamline the paperwork process, this new collection of surveys replaces unexpired collection Federal Emergency Management Agency Individual Assistance Survivor Centric Customer Satisfaction Survey 1660-0129. The survey, which expires 2/28/2018, will be replaced with this new collection. Upon approval of this new collection, the current collection will be discontinued.

B. Collections of Information Employing Statistical Methods.



When Item 17 on the Form OMB 83-I is checked “Yes”, the following documentation should be included in the Supporting Statement to the extent it applies to the methods proposed:

If the collection does not involve statistical methodology please enter “THERE IS NO STATISTICAL METHODOLOGY INVOLVED IN THIS COLLECTION” and delete Q1 through 5.

1. Describe (including numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

The sampling frame consist of all disaster survivors who registered for assistance with FEMA. The system is flagged by the preferred communication method and the administration method to deliver the survey (Phone or Electronic).

1. Phone Survey: Historical data shows approximately 68% applicants with a preferred communication method by US Mail with a response rate of approximately 28%.

2. Electronic Survey: Historical data shows approximately 32% applicants with a preferred communication method of email. The expected response rate for the electronic survey is approximately 30% based on empirical research of web based surveys and industry standards.

The subpopulations are then grouped by the intent of the scope of each survey as follows:

Preparedness Survey (Phone or Electronic) measures the preparedness levels of FEMA applicants. The sample will include a proportionate amount of eligible applicants by disaster.

The total yearly population of applicants is approximately 125,049, based on a 6 year average of disaster registration data. The target number of completions per quarter is 400 to ensure statistical inference with 95% confidence level, 0.5 variability assumption on the population, and 5% precision (Margin of error).

 

Transitional Sheltering Assistance Survey (TSA) (Phone or Electronic) measures the quality of disaster assistance information and service received regarding eligibility and availability of hotel accommodations for disaster survivors. The sample will include a proportionate amount, by disaster of TSA eligible applicants who participated in the program. The total yearly population of applicants is approximately 54,357, based on a 6 year average of TSA eligible data. The target number of completions per quarter is 400 to ensure statistical inference with 95% confidence level, 0.5 variability assumption on the population, and 5% precision (Margin of error).

Temporary Housing Units Survey (THU) (Phone or Electronic) measures the quality of disaster assistance information and service received regarding eligibility and availability of housing units for disaster survivors. The sample will include a proportionate amount, by disaster, of THU eligible applicants who participated in the program.

The total yearly population of applicants is approximately 23,943, based on a 6 year average of THU eligible data. The target number of completions per quarter is 400 to ensure statistical inference with 95% confidence level, 0.5 variability assumption on the population, and 5% precision (Margin of error).

Shelter and Temporary Essential Power Survey (STEP) (Phone or Electronic) measures the ease of understanding information received, timeliness, and expectation and satisfaction the temporary repair process of the STEP program. The sample will include a proportionate amount, by disaster, of eligible applicants who participated in the STEP program. . Services provided through the program vary based on disaster specific needs and State partner agreements. The State may also refer to the program under a different name when establishing and advertising the program to their applicants.

The total yearly population of applicants is approximately 14,186, based on 2 years of STEP program participant data. The target number of completions per quarter is 400 to ensure statistical inference with 95% confidence level, 0.5 variability assumption on the population, and 5% precision (Margin of error).

Due to the infrequency of the TSA, THU, and STEP programs, data from the past 6 years were averaged and used when available.

Qualitative research (focus groups and interviews) will not be subject to probabilistic sampling methods (e.g., usually based on purposive or convenience sampling). Historical data shows response rates of 6% for focus groups without incentive to participate.

Part B Question #1: Description of Respondent Universe, Sampling Method, Response Rates
Type of Respondent / Entity	Form Name / Form Number	Potential Respondent Universe Numerical Estimate	Target Completions per Quarter	Target Completions per Year	Sampling Criteria for Universe	Actual or Expected Survey Response Rates	Target Annual Adjusted Sample Size
Surveys		A	B	Bx4 Qtrs.		D
Individuals and Households	Preparedness Survey- Phone	85,033	272	1,088	quarterly Sample proportionate to applications per disaster	28%	3,886
	FEMA Form 519-0-44
	Preparedness Survey- Electronic	40,016	128	512	quarterly Sample proportionate to applications per disaster	30%	1,707
	FEMA Form 519-0-45
	Preparedness Survey	125,049	400	1,600			5,592
	TSA Survey- Phone	32,110	272	1,088	quarterly Sample proportionate to applications per disaster	28%	3,886
	FEMA Form 519-0-46
	TSA Survey-Electronic/	15,110	128	512	quarterly Sample proportionate to applications per disaster	30%	1,707
	FEMA Form 519-0-47
	TSA Survey	47,220	400	1,600			5,592
	THU Survey- Phone	1,088	272	1,088	quarterly Sample proportionate to applications per disaster	28%	3,886
	FEMA Form 519-0-48
	THU Survey-Electronic/	512	128	512	quarterly Sample proportionate to applications per disaster	30%	1,707
	FEMA Form 519-0-49
	THU Survey	1,600	400	1,600			5,592
	STEP Survey- Phone	9,987	272	1,088	quarterly Sample proportionate to applications per disaster	28%	3,886
	FEMA Form 519-0-50
	STEP Survey-Electronic/	4,700	128	512	quarterly Sample proportionate to applications per disaster	30%	1,707
	FEMA Form 519-0-51
	STEP Survey	14,687	400	1,600			5,592
Total Survey Sample Size		188,555	1,600	6,400			22,370
Qualitative Studies
Individuals and Households	Focus Group for 2 Hrs Plus Travel 1 Hr	125,049		960		6%
	One-on-One Interviews	125,049		768
	On-Line Interviews	125,049		768
Qualitative Studies Total				2,496
Surveys and Qualitative Studies				8,896

The table below shows the estimated size of the universe covered by the collection and the corresponding samples for each survey.

2. Describe the procedures for the collection of information including:

-Statistical methodology for stratification and sample selection:

A probability based sampling method of stratification will be used to make sure each disaster within the population timeframe is represented with proportionate estimates. These disaster subgroups will be based on proportionate registrations by disaster.

“Simple random samples (where all units and all equal-numbered combinations of units have the same probabilities of selection) are rare in practice for a number of reasons. … Thus, other probability-based methods that employ multiple stages of selection, and/or stratification, and/or clustering are used to draw more practical samples that can be generalized with known degrees of sampling error.” [https://www.whitehouse.gov/sites/default/files/omb/inforeg/pmc_survey_guidance_2006.pdf]

Stratification provides gains in precision, or reliability, of the survey estimates and the gains are greatest when the strata are maximally heterogeneous.

-Estimation procedure:

For the Preparedness Survey, the sample is based on the FY2016 response rates of previous similar surveys from an old collection. Sample is adjusted to accommodate historical response rates to improve reliability.

For the TSA, THU, and STEP surveys, the sample is based on historical response rates of FEMA satisfaction surveys with similar lengths of time between disasters.

-Degree of accuracy needed for the purpose described in the justification:

The degree of accuracy is obtained by using a 0.5 variability assumption on the population (response distribution), 5% precisions (Margin of error) and 95% confidence level. This sample size allows us to make statistical inference of the population.

-Unusual problems requiring specialized sampling procedures:

There are no unusual problems requiring specialized sampling procedures.

-Any use of periodic (less frequent than annual) data collection cycles to reduce burden:

Due to the nature of STEP, TSA, and THU, these programs may not activate to be available with every disaster. Since the program occurrences are not predictable enough to schedule a collection cycle in advance, data may be collected and reported quarterly, as available.

STEP, TSA, and THU surveys ask questions related to their overall experience of the program services, after the disaster occurred; therefore, the survey is conducted 30 days or more after the conclusion of program for the applicant or close of the application period. The Preparedness Survey asks questions related to preparedness levels and information which need more time to experience after the disaster occurred; therefore, the survey is conducted 90 days or more after the disaster.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Maximizing Response Rates

Maintaining adequate response rates of surveys continues to be a problem as more people are fatigued from survey inundation, and highly publicized confidentiality breaches from various organizations have people uneasy about providing information. Because of this, several methods are used to maximize response rates: Survey burden time is short, and mixed mode administration is offered (phone & electronic).

Reliability and Validity (Accuracy)

Sample is adjusted to accommodate historical response rates to improve reliability. This is done by taking similar surveys’ response rates and increase the targeted completions. If we would like 400 completions but we know we only receive 50% a response rate, we would then survey 800 to ensure we receive the 400.

Questions are screened to ensure readability through research of best practices and read aloud testing. Response options are also screened to create independent/ non overlapping options and dubious replies due to unclear or overlapping response scales. Complex wording, technical terms, jargon, and difficult phrases are closely monitored.

Unreliable Data

Factors that contribute to the non-response portion may be due to the nature of the disaster; such as, due to the disaster applicants who are survivors often do not have telephone service, cell phone service, nor electrical service in their community. Frequent relocations and displacements are anticipated affecting the respondent’s availability to complete the survey. Survivors may not want to use their cell phone minutes to respond to a survey. Disaster trauma may be a factor as the survivor may not remember all the different interactions with FEMA or was not familiar with the case. Due to these factors, sample size is adjusted to accommodate historical nonresponse rates to alleviate possible unreliable data.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

FEMA personnel reviewed the questionnaire content and wording to improve readability and clarity.

Tests for readability are conducted by staff to help with reliability and accuracy. This includes question layout, wording, definitions, and timing. Questions are also analyzed for plain language.

Tests with fewer than 10 survivors may be performed by FEMA’s Customer Survey and Analysis staff for updates or revisions as needed.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The Customer Survey & Analysis (CSA) Section plans, designs, administers, and analyzes results of the survey. This includes the survey methodology and sample selection, collecting, tabulation and reporting of the data.

Jessica Guillory, Statistician

Customer Survey & Analysis

Federal Emergency Management Agency

940 891 8528

Gena Fry, Program Analyst

Customer Survey & Analysis

Federal Emergency Management Agency

940 891-8543

Chad Faber, Supervisory Program Analyst

Customer Surveys and Analysis

FEMA Recovery Reports and Analytics

(940) 891-8956

22