Part B of SS

B. Statistical Methods (used for collection of information employing statistical methods)

1. Respondent Universe and Sampling Methods

The Annual Survey of Refugees is a longitudinal panel study, tracking participant characteristics and outcomes over a five year period. Each year, a sample of refugees and Cuban/Haitian entrants (as well as certain other eligible populations of interest) is drawn from ORR’s Refugee Arrivals Data System (RADS). The methodology for drawing the sample is as follows:

• The RADS contractor creates a table of arrivals (refugees and Cuban/Haitian entrants, as well as certain other eligible populations of interest) with date of arrival in the U.S. between May 1^st of the prior fiscal year and April 30^th of the current fiscal year. Each population being sampled is extracted into a separate table.

• This date range is chosen in order to meet the required due dates for the Annual Report to Congress.

• The sampling frame, and therefore the sample, consists of all persons who have entered the United States as refugees, Amerasian immigrants (including their immediate family members), Cuban and Haitian entrants, and possibly certain other eligible populations of interest, during the specified time period.

• If ORR wishes to obtain information on a certain group within this population (such as refugees arriving from a particular country) or another eligible population of interest (e.g., a newly eligible population such as Iraqi or Afghan Special Immigrants), ORR may request that the RADS contractor sample individuals with these characteristics at a higher rate than the general sample (i.e., oversample them). This allows the Annual Survey contractor to gather more information on the outcomes of this particular group of arrivals.

• Each individual in the table is given an individual identifier (e.g., between 1 and 70,000).

• ORR provides the RADS contractor with a desired sampling ratio, based on the number of arrivals in the table and the required size of the sample (e.g., 1:250 ratio).

• The RADS contractor generates a random sample of the individuals in the table, using the random number generator feature in Microsoft Excel. Once this is complete, the RADS contractor generates a second random sample of the individuals in the table using the same methodology. Duplicates (individuals who appear in both files) are removed.

• The sample cohort files are provided to the Annual Survey contractor so that they can begin tracing and interviewing respondents.

Each sample cohort of refugees is interviewed annually for five years. A longitudinal panel of refugees and entrants who arrived in past fiscal years already exists, and (2) a new sample of individuals who entered the United. In each year after thereafter, the sample cohort of individuals who entered the United States in the earliest year will be dropped, and a new sample from the current year representative of all refugees and entrants will be added. Each year's cohort of refugees will remain in the sample for five years, participating in up to five interviews.

In the past several years, the response rate has ranged between 25 and 35 percent. as a result of improved tracking and use of technology, the response rate rose to 50.3 percent. Though a response rate of 50 percent may seem low compared to surveys of the broader population, we believe that it should be considered a very good response rate for the hard-to-reach, often mobile refugee populations surveyed. Many refugees are wary of responding to these kinds of sensitive surveys, and often change their addresses as they become settled in the U.S. Many also lack phone numbers or easy ways to contact them.

2. Procedures for the Collection of Information

The information collection procedure is as follows:

• Tracing of Respondents

  • The contractor shall rely on existing sample files for re-contacting the prior cohorts of persons interviewed or approached for earlier survey interviews. Newly sampled respondents are traced and contacted by letter, by telephone, or both. Extensive tracing activity may be required for some respondents. The contractor is expected to work with local and national voluntary agencies to locate respondents when necessary.

• Advance Mail-out

  • The contractor prepares and sends a Request for Participation letter (included in clearance package) to each of the potential respondents. The letter introduces the survey and requests a prompt return of a pre-addressed, postage-paid letter. The letter will provide the potential respondent the opportunity to indicate his/her willingness to participate in the survey and to furnish the contractor with his/her current telephone number. If a potential respondent does not return the letter, the contractor attempts to obtain the telephone number and obtain the individual’s participation using other available means, such as contacts in the local community. Extensive efforts are taken in order to achieve the highest response rate possible.

• Translation of Survey Instrument

  • The contractor obtains translations of the survey instrument described in this information collection into various foreign language versions spoken by refugee populations.

• Interviewing

  • The contractor contacts each respondent who has indicated his/her willingness to participate in the survey. Refugees are not interviewed if they have been in the United States for less than five months the sampling frame, and therefore the sample, consists of individuals who have entered the United States during the specified time period. A large proportion of the refugee population consists of children, so many of those sampled are children. For both practical and cultural reasons, if the sampled person is a child or young person, the contractor will interview the head of the household or other responsible adult who agrees to serve as the respondent. The relevant household is defined as the household in which the sampled person is living at the same time of the interview; the interview schedule pertains to the household as an entity and to all adults living in the household. This has two implications: (1) the household may change composition over the five years that the selected person is in the sample; (2) the household may contain members who arrived in the United States more than five years previously; the interview is to cover them as well.

  • Generally, interviews are conducted over the telephone. In cases where telephone interviewing is not feasible or desired, and in locations (as determined by the contractor in consultation with the project officer) where the survey population is large enough to make it cost-effective, the interviewer may interview participants in-person at a location of the participant’s choice (we do not expect this to occur frequently). The contractor shall attempt to conduct the survey in the appropriate native language of the respondent. In addition to the languages translated in the survey instrument, the contractor may hire a translator to accommodate additional languages for some interviews. As each oral interview progresses, the interviewer completes the hard copy or electronic survey instrument which shall be the prime data gathering instrument.

• Quality Control

The contractor:

• Prepares a questionnaire reference book, in consultation with the project officer, for use by the interviewers. The contractor provides training to the interviewers in the conduct of these interviews in order to minimize interviewer error and variability. Prior to interviewing, an intensive three-part training session is developed to prepare the interviewer for the survey season and to minimize error and variability.  Interviewers will receive a thorough explanation of each survey question and identify logical and acceptable responses to questions;

• Ensures that interviewers employ the most up-to-date interviewing techniques to minimize the length of the interview, including use of computer assisted telephone interviewing (CATI) methods, if applicable;

• Downloads and reviews data tables from CATI system

• Runs a series of frequency tests to identify any erroneous anomalies;

• Deletes incomplete records, if required;

• Identifies and corrects missing data (missing data are not likely as the interviewing is automated. Questions and answer choices follow logical skips patterns);

• Determines the measurement level for each data field (i.e. string or numeric);

• Checks and deletes unexpected symbols in the data fields (e.g. “,” “+”);

• Identifies and deletes duplicate records;

• Checks the response categories of each categorical variable against the original questionnaire;

• Attaches correct variables and value labels to each data field;

• Conducts logic checks to identify correct outliers (i.e. out of range values);

• Checks skip patterns and assign appropriate value labels to not-applicable;

• Determines the difference between the missing values and ‘0’ values, which is particularly important for calculating means; and

• Edits the verbatim responses to ensure that the recorded answers are meaningful.

3. Methods to Maximize Response Rates and Deal with Non-response

 

The contractor relies on sample data provided in earlier survey years to contact persons previously interviewed. All sampled persons are traced and contacted by letter, and telephone, or both. Should these methods yield minimal results, we attempt additional tracing activities including, but not limited to:

 

• The usage of internet people search sites;

• Working with reliable Database Warehouses;

• Seeking information from non-profit Voluntary Agencies (VOLAG) that are involved in the resettlement of refugee populations;

• Sending certified mail;

• Translation of Survey Instrument into respondent’s native language; and

• Comprehensive training of the research interviewers. Prior to interviewing, an intensive three-part training session will be developed to prepare the interviewer for the survey season and to minimize error and variability.  Interviewers will receive a thorough explanation of each survey question and identify logical and acceptable responses to questions.

 

Non-response procedure: Sampled persons who are not interviewed by the end of the survey period are compiled into a separate database. This non-response database is maintained by HHS.

 

4. Test of Procedures or Methods to be Undertaken

The data collection procedures and instruments are virtually identical to ones used previously with satisfactory results.

 

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Wanda Hall

Office of Refugee Resettlement

Administration for Children and Families

Mary E. Switzer Building, 5th Floor

330 C Street, SW, Washington, DC 20201

(202) 401-5409

wanda.hall@acf.hhs.gov

Joann Simmons

Director of Budget

Office of Refugee Resettlement

Administration for Children and Families

Mary E. Switzer Building, 5th Floor

330 C Street, SW, Washington, DC 20201

(202) 401-5409

joann.simmons@acf.hhs.gov

 

Gary Frost

Program Manager

Civilian and Homeland Security Solutions Division 

General Dynamics Information Technology

3211 Germantown Road

Fairfax, VA  22030

(703) 995-3700 direct

gary.frost@gdit.com

www.gdit.com