The respondent universe is FBCOs located in Louisiana, Mississippi, and Houston that were in critical geographic areas for mounting relief/recovery efforts after hurricanes Katrina and Rita, namely: (a) FBCOs in the directly impacted areas of the storm; (b) those in areas adjacent to or near the impacted areas; and (c) those away from the impacted areas but still in the states of Louisiana and Mississippi, and in Houston, one of the major evacuee receiving areas. We will use the extent of postal service disruption as a proxy for defining and operationalizing these concepts. For example, zip code areas in which postal service was totally disrupted and mail service was not available will be regarded as the directly impacted areas. Zip code areas in which mail service was partially available after the storm will be considered the nearby areas, and zip code areas that were within Louisiana, Mississippi, and Houston but had no interruption of mail service will be considered the outer boundaries of the study. Information on zip codes will be derived from the U.S. Postal Service, “Impact of Hurricanes Katrina and Rita on Post Office Operations.”
We will draw a sample of FBCOs from this universe for the telephone survey and case studies. This design will enable the study to investigate various types and extent of assistance that FBCOs conducted relative to their geographic proximity to the storm and the networks that were used as part of the response effort. The plan proposed below is designed to meet rigorous research standards within the time and budget constraints of the study.
Because there is no master list of FBCOs in the geographic region of interest, it is necessary to create one. Given the outpouring of contributions and assistance that flowed into the Gulf Coast region after the storm, it is impossible to generate a complete list of all organizations that responded to the crisis. We therefore will focus on identifying FBCOs from the two states most impacted by the storms (i.e., Louisiana and Mississippi), plus one of the major evacuee receiving areas (namely, Houston) as the basis for the telephone survey.
To create a relatively complete list of FBCOs, we will use two primary sources, crosschecked with lists of FBCOs obtained by our local partners. The first source will be the American Church List. The American Church List is one of the most widely used sources for obtaining directories of congregations and houses of worship in the United States. We will purchase a list of congregations and houses of worship in Louisiana, Mississippi, and Houston. This database can be sorted by geographic area to pinpoint locations of interest. The second source will be the Urban Institute’s National Center for Charitable Statistics (NCCS) database. The NCCS database contains all nonprofit organizations with annual gross receipts of $25,000 or more, which file a Form 990 with the Internal Revenue Service. We will create a list of nonprofit human service community-based organizations in Louisiana, Mississippi, and Houston from the NCCS database.
By purchasing lists of congregations and houses of worship from the American Church List and using the information on community-based, human service nonprofits in the NCCS database, we will have a reasonably complete list of FBCOs in the geographic areas of interest (i.e., Louisiana, Mississippi, and Houston), which will enhance our ability to generalize to the larger population for this region. These lists contain information on FBCOs of various sizes and cover most Judeo-Christian faiths.1
In addition, our local partners (the Louisiana Association of Nonprofit Organizations and the Mississippi Center for Nonprofits) will obtain lists of FBCOs that participated in the relief/recovery efforts from various other sources, such as the governor’s office and relief agencies. These locally obtained lists will be crosschecked with the American Church List and NCCS list to improve the completeness of the master list and reduce potential coverage bias. Duplicate entries will be discarded so that each FBCO appears only once in the master list. The lists obtained by our local partners may also provide us with additional contact information for FBCOs that can be matched across lists.
Stratification decisions will focus on three main critical elements that are likely to affect the types of relief services provided and the networks used in providing services: (1) the geographic location of the responding FBCO (i.e., distance from the areas of the storm’s immediate impact); (2) the size of the FBCO; and (3) religious denomination or affiliation or type of organization. All three elements may be important for understanding the types of responses, level of effort, and structure of relief/recovery networks in the aftermath of the storms. The sample selection criteria will be limited to up to three of these factors, so the subsample sizes are not too small for statistical analysis. When the master list is complete, we will run frequency distributions on the entries to determine the number of cases that contain any or all of the critical elements. We will also review the critical elements to determine the range and specificity of the information.
Because the proposed lists will contain an address or telephone contact, we will be able to stratify by geography. More problematic will be obtaining sufficient information on size, organization type, and denomination to use these elements as stratification criteria. We will determine how many FBCOs in the list have provided their religious denomination and how many specific denominations are named. Based on this analysis, we will be able to determine: (a) if there is a sufficient number of cases with this type of information to use the element as a stratification criterion; and (b) if we are able to group the information into logical categories (such as, small, medium, and large organizations). In the event that we cannot stratify by size, type, or denomination, we will ask for this information as part of the telephone survey so we will be able to include these criteria in the analysis.
The sampling methods for the telephone survey and the case studies are described separately.
Telephone Survey. The sample design for the telephone survey calls for a random sample of FBCOs in Louisiana, Mississippi, and Houston, stratified by geography. As indicated above, we may expand the stratification criteria to include size of the FBCO, or religious affiliation, if there are a sufficient number of cases with this type of information in the master file. The resulting sample will enable us to generalize to the broader population of FBCOs in our universe.
Because we do not have pre-existing information on the universe of FBCOs that provided relief/recovery services after the storms, the sampling strategy will depend, in part, on the distribution of organizations that are reflected in the master list. At a minimum, we plan to stratify by geography using state and zip code as the basic stratifying criteria. Zip codes will be classified into three groups: (1) those entirely within the directly impacted areas; (2) those that touch on or are partially in the directly impacted areas; and (3) those outside the impacted areas, but in other parts of Louisiana and Mississippi, and in Houston. Zip codes can also be used to approximate rural, urban, or mixed (rural/urban) locations.
We will initially draw a sample of 600 FBCOs that will serve as the sampling pool for the telephone survey. Our goal is to achieve approximately 200 completed interviews, with a margin of error due to sampling at the 95 percent confidence interval for any estimated proportion based on the total sample, that is, at most ±5 percent. The strategy for achieving this response rate is discussed under section B.3, below.
To select the 600 FBCOs, we will develop sampling criteria to reflect the universe of the master list. For example, if 60 percent of the FBCOs in the list are from Louisiana, we will randomly select 60 percent of the sample (i.e., 360 FBCOs) from FBCOs with a Louisiana address. The greater the number of sampling criteria (such as sampling by size and geographic location), the smaller each cell size will be for specific criterion (e.g., for small FBCOs in Louisiana). We therefore plan to limit the selection criteria to no more than two or three factors. Stratifying by size and denomination/affiliation will be undertaken if the information on the master list is sufficient to allow reasonable classification along these dimensions. If we cannot stratify by size and/or denomination/affiliation, we will obtain measures of these factors in the telephone survey.
Case Studies. The selection of FBCOs for the case studies will be based on preliminary information derived from the telephone survey and feedback from our local advisors and TAG. The cases will be selected to illustrate different types of response models in order to explore the ways in which different organizational types and networks functioned in the delivery of relief services and to suggest good practices for effective disaster relief. While there are too few case studies to provide a statistically or nationally representative sample, we will strive for a sample that gives a picture of a variety of experiences encountered. For example, in selecting the case studies, we will look for FBCOs that illustrate differing sizes, functions, cultures and missions, and levels of affiliation, and ones that operate in urban or rural settings, and serve communities of different racial/ethnic or socio-economic groups.
Telephone Survey. An introductory letter (Attachment F) will be sent from the Public Policy Research Lab (PPRL) at Louisiana State University to executive directors and clergy of the 600 FBCOs that were selected in the sample. The letter describes the purpose of the survey, the authority for data collection, that participation is voluntary, and that information collected in the survey will be treated as private within the limits of the law. The recipient will be given contact information at the PPRL and Urban Institute, if they desire additional information about the study.
Case Studies. Selected case study organizations will be sent a letter (Attachment G) informing them of the study and requesting their participation. FBCOs will then be contacted to arrange the local site visit. The initial telephone contact will provide background about the project and seek additional information on organizations and partners in order to identify key respondents. Based on this information, we will contact respondents and determine the best timing for the visit in order to accommodate the schedule of local respondents.
Telephone Survey. PPRL will assign a team of professional interviewers to this study to conduct the data collection activities. Before data collection begins, the interviewers will receive training specific to the project and survey instrument. The training will focus on the objectives of the study and on procedures to follow in difficult interviewing situations. PPRL will go over the entire questionnaire with the interviewers and explain the purpose of the questions, definitional issues, and how to handle respondent confusion or ambiguous responses and skip patterns, and how to code the answers. The Urban Institute will oversee the preparation and delivery of the interviewer training; PPRL will conduct the training sessions, which are expected to take approximately three-four hours to complete.
Case Studies. The case study site visits will be conducted by two-person teams drawn from Urban Institute staff and composed of one senior and one junior staff member. Senior staff on this project are experienced in field-based qualitative research and semi-structured interviewing of the type that will be used in this study. All Urban Institute staff involved in the fieldwork will be trained with respect to the objectives of the study and the procedures to follow during the site visits. In the training, team members will review the different discussion guides, become familiar with the types of information sought in the study, and, through role playing, practice their interviewing, listening, and note taking techniques. It is anticipated that the training will take approximately four hours.
Telephone Survey. To ensure quality control during the survey’s data collection phase, PPRL supervisors will closely monitor the initial interviews and help the interviewers with any unanticipated problems, including review of how to handle problematic responses, which may arise in the first few days of fielding the survey. After the initial field period, PPRL will monitor interviews intermittently and review field progress weekly.
During the data collection phase of the study, PPRL will send weekly updates via email to the Urban Institute on the number of FBCOs that had been contacted, the number of interviews scheduled, the number of interviews completed, and the number of hard and soft refusals. These updates will be used to monitor the completion or response rate and to make adjustments, if necessary. Communication between PPRL and Urban Institute will also be maintained through frequent phone calls, and the HHS-ASPE Project Officer will be updated regularly on the progress of the survey.
Case Studies. Prior to visiting sites or speaking with any potential respondents, Urban Institute staff will review all available materials about the local site and the selected FBCO. This will enable us to identify the appropriate individuals to interview on site who can best inform the central questions in the study. We will also rely heavily on our local partners, TAG members, and other informants to guide this work.
As noted earlier and is common practice with field-based research, project staff will produce detailed notes of their interviews and a full site summary of each case study, both of which are reviewed by fellow team members to ensure that gaps or inconsistencies are resolved in a timely fashion, and the data are reliable for analysis and production of briefing memoranda and the final report.
Telephone Survey. Prior to the start of data collection, the 600 selected FBCOs will be prescreened to determine if they still exist, participated in the relief effort, and can be contacted by telephone. Various efforts will be made to obtain up-to-date contact information, including Internet searches, directory matches, and possibly reverse matching services. If after repeated effort we cannot verify the existence of the group or secure a current contact number, we will replace the FBCO in the sample by using the same random selection procedure described above. In addition, we will also replace at random those organizations that said they did not participate in the relief effort.
We will use a 10-call design for the survey—that is, a minimum of 10 attempts will be made to contact every sampled FBCO, staggered over different times of the day and days of the week to maximize the chance of making contact with the FBCO. If an individual cannot answer the survey immediately, the interviewer will schedule a time to call back to complete the survey. A 10-call attempt should be more than adequate since most of the sampled organizations will have been pre-qualified with updated contact information.
Several techniques will be used to facilitate a high response rate and achieve 200 completed interviews. First, although we expect to draw a sample of approximately 600 FBCOs, we will not release all of the names and contact information at one time. Once we have completed the prescreening process, we will randomly divide the 600 into 12 release groups (i.e., 50 FBCOs per group) and release the sample in waves, starting with the first 5 groups, and tracking the number of completed interviews, scheduled interviews, refusals, and incorrect/inoperable telephone numbers. This will enable us to monitor the projected response rate. As needed, additional names will be released from the pool of 600 FBCOs until we complete the target of 200 respondents. Depending on the contact and response rates during data collection, it is possible that the study could be completed with as few as five release groups, but most likely we will need all twelve release groups to complete the data collection. Releasing the sample in waves increases the likelihood of achieving a high response rate because the response rate is sensitive to the size of the released sample (that is, the denominator); we will use the smallest sample possible to generate the targeted number of completed questionnaires.
Second, PPRL at LSU will send an advance letter to the pool of 600 FBCOs summarizing the purpose, sponsorship and other relevant details to encourage participation and provide legitimacy for the survey. Although some of these letters may not reach their intended recipients because of continuing mail disruption in the most affected areas, we believe that the majority of FBCOs will be reached in this way to notify them of the study. Additionally, PPRL will make available to FBCOs its toll-free 800-number to schedule interviews at times that are convenient for the FBCOs. Although many surveys of FBOs achieve relatively low response rates (typically 20-30 percent and generally, less than 50 percent), we have found from our interactions with service providers and others in Louisiana since the storms that they are eager to share their experiences and are grateful for an opportunity to tell their stories to outsiders. We therefore anticipate a relatively high level of cooperation and expect a response rate of 80 percent.
Case Studies. For the case studies, it is expected that all (or nearly all) of the FBCOs we approach will agree to participate in the study. We will work closely with our partners in Louisiana and Mississippi to engage these FBCOs and assuage any concerns about participating in the study. Once we have secured the selected sites, site visitors will work closely with a person assigned to be the primary contact at the FBCO to help in scheduling the site visit. One member of the two-person site visit team will take responsibility for working with the primary contact person to handle the scheduling and logistics of the site visit. For the site visits, the logistical discussion will include recruitment of focus group participants and arranging a time and location for the focus groups. Dates for site visits will be made at least one month ahead of time to permit ample time to schedule interviews. Scheduled interview appointments will then be confirmed via email the week prior to the visit. We will request that a quiet setting that is as private as possible (e.g., a conference room) be made available to interview those who do not have private offices, in order to encourage respondents to feel they can talk freely. Based on our experience, following these established field visit protocols leads to an interview completion rate approaching 100 percent of those scheduled in advance.
To ensure that the desired number of people attend the focus group sessions (i.e., 6-10 people), we will need to over-recruit to allow for the incidence of no-shows. We estimate that we will need between 16-20 people to agree to attend a focus group session to ensure a group of sufficient size. We will work closely with our site contact to recruit focus group participants and provide the contact with the necessary information. A small monetary incentive will be offered to potential focus group participants to help encourage their participation. Once focus group candidates are identified, they will be contacted by telephone to fully inform them about the purpose of the focus group and determine whether they are willing to participate. These candidates will be called by telephone two-three days prior to the focus group to remind them about the session and address any outstanding questions or concerns.
The estimated response time for the telephone survey is based upon a pretest conducted with 7 FBCOs located in the Gulf Coast region. The range of time to complete the survey was 15 to 25 minutes. We believe that the range in time reflects the size of the FBCO as well as the extent of automated data systems. A few minor revisions were made to the survey questionnaire to limit the response time to no more than 20 minutes, on average, to complete.
Telephone Survey. PPRL pre-tested the telephone survey instrument, using seven FBCOs in the Gulf Region: five in Louisiana, one in Mississippi, and one in Houston. The FBCOs were of different sizes with budgets ranging from less than $10,000 to $5 million. Four respondents were faith-based organizations and three were community-based nonprofits. The seven FBCOs participating in the pretest were:
Harvest Ripe Church; 423 Amelia Street; Gretna, LA 70053
Accountability Church; 33719 La Highway 16; Denham Springs, LA 70706
First Church of God; 210 South Briggs Street; Oak Grove, LA 71263
Alexandria Electrical Joint Apprenticeship Committee; 6703 Masonic Drive; Alexandria, LA 71301
Houma-Terrebonne Civic Center Development Corporation; 346 Civic Center Blvd.; Houma, LA 70360
Silver Cross Home; 503 Silver Cross Drive; Brookhaven, MS 39601
Trinity Lutheran Church; 800 Houston Ave; Houston, TX 77007
Results of the pretest were used to refine the telephone survey instrument and study procedures. The results of the pretest were submitted to HHS/ASPE on June 14, 2007.
Case Studies. Case study instruments were reviewed for content, methodology, and burden estimate by our TAG members. The instruments have been revised to reflect comments by these reviewers and the research team, who have conducted many similar studies. Overall, reviewers report that the discussion guides capture the intended data and in the prescribed amount of time to minimize burden on respondents.
The agency responsible for funding the study, determining its overall design and approach, and receiving and approving contract deliverables is:
Office of the Assistant Secretary of Planning and Evaluation
U.S. Department of Health and Human Services
200 Independence Ave. SW, Room 404-E
Washington, DC 20201
Person Responsible: Alana Landey (phone: 202-401-6636; email: Alana.Landey@HHS.GOV)
The Urban Institute is the prime contractor for this study. It is responsible for implementing the overall design of the study and development of the data collection instruments. It will field the case studies using its own staff, and will have responsibility for all data analyses obtained through the telephone survey, case studies, and focus groups.
The Urban Institute
2100 M Street, NW
Washington, DC 20037
(202) 833-7200
Persons Responsible: Carol De Vita and Fredrica Kramer, Co-Principal Investigators
Timothy Triplett, Survey Associate and Statistical Expert
Direct Contact Information:
De Vita (phone: 202-261-5232; email: cdevita@ui.urban.org)
Kramer (phone: 202-261-5399; email: fkramer@ui.urban.org)
Triplett (phone: 202-261-5579; email: ttriplett@ui.urban.org)
The Public Policy Research Laboratory at Louisiana State University is a subcontractor in this study. It will field the telephone survey and provide a raw data file to the Urban Institute.
Public Policy Research Laboratory
Louisiana State University
Baton Rouge, LA 70803
Person Responsible: Steven Procopio (phone: 225-578-7499; email: sproco1@lsu.edu)
The Louisiana Association of Nonprofit Organizations (LANO) and the Mississippi Center for Nonprofits (MCN) will serve as local partners with the Urban Institute team. These organizations will secure local lists of FBCOs that participated in the post-hurricane relief/recovery efforts and help identify potential FBCOs for site visits during the case study phase of the project.
Persons Responsible: Susan Hymel and Dorothy Thomas, both of LANO
Susan Weifhar, MCN
LANO Mississippi Center for Nonprofits
P.O. Box 3808 700 North Street – Suite 201
Baton Rouge, LA 70802 Jackson, MS 39202
Director Contact Information:
Susan Hymel (phone: 225-925-2390; email: susan@lano.org)
Dorothy Thomas (phone: 703-581-3989; email: Dorothy@lano.org)
Susan Weifhar (phone: 601-968-0061, ext. 16; email: sue@msnonprofits.org)
1 The sample is likely to under-represent non- Judeo-Christian faiths, however we believe this will be a small bias for the geographic area of interest. When possible, we will supplement the American Church List with information obtained by our local associates regarding non-Judeo-Christian houses of worship.
| File Type | application/msword |
| Author | DHHS |
| Last Modified By | DHHS |
| File Modified | 2007-07-13 |
| File Created | 2007-07-13 |