SS_B REACH US Eval Revision

Racial and Ethnic Approaches to Community Health across the U.S. (REACH US) Evaluation

OMB No. 0920-0805

Request for Revision

Supporting Statement Part B

January 12, 2012

Project Officer: Youlian Liao, MD

Division of Adult and Community Health (DACH)

National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP)

Centers for Disease Control and Prevention (CDC)

Atlanta, GA 30341

Telephone: (770) 488-5299

Fax: (770) 488-5974

Email: YCL1@cdc.gov

Table of Contents

B. Collections of Information Employing Statistical Methods............................... 3

1. Respondent Universe and Sampling Methods.............................................. 3

2. Procedures for the Collection of Information............................................... 3

3. Methods to Maximize Response Rates and Deal with Nonresponse.......... 5

4. Tests of Procedures or Methods to be Undertaken...................................... 5

5. Individuals Consulted on Statistical Aspects and Individuals

Collecting and/or Analyzing Data................................................................. 6

B. Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

Each REACH U.S. grantee has a defined a geographic region for intervention activities and data collection. For each community, the contractor, the National Opinion Research Center (NORC) at the University of Chicago, will be using an address-based sampling (ABS) approach to select respondents. The source of the ABS frame is the US Postal Service delivery sequence file (DSF). The DSF contains nearly all addresses in the United States that receive mail, including city-style addresses, P.O. Boxes, and rural-route addresses. While the DSF also contains business addresses, we will only be using the residential portion of the file for our REACH sampling purposes. NORC will use geographic information systems (GIS) technology to geocode the DSF and so determine the longitude and latitude for each address. In this way it is possible to target very specific geographies, down to the census block level, for those addresses that are city-style. The target geography is translated into census blocks and a sampling frame is constructed from which the sample is drawn. To increase sampling efficiency, for some communities NORC will stratify the non-overlapping target geography into strata based on the density of the targeted minority populations.

We oversample middle-aged women in REACH communities that focus on breast and cervical cancer and oversample older adults in communities that focus on adult immunization. For these communities we will also be using supplemental lists provided by the vendor to target certain types of households, specifically those with members in particular race/ethnicity, age, and gender categories. The vendor maintains a comprehensive household file which is created by aggregating data from numerous sources, including residential directory listings, administrative data, and consumer transactions. When two frames (one general frame and another race/ethnic, age/gender-targeted frame) are needed for one community, we will remove the targeted list frame from the general frame to create two completely unique and non-overlapping frames from which to select the sample in each community. Each eligible adult could be selected into the study sample once and only once from each frame.

NORC will assign the sampled addresses to replicates each containing 100 addresses. Each replicate is a random subsample of the original sample of addresses. Initially, approximately 50 percent of the sample NORC expects to use will be released and worked. Replicates will continue to be released as needed for data collection to manage the workload and to achieve the targeted number of completed cases with a minimum of excess. Monitoring reports will be reviewed during data collection to assure the optimal number of replicates is used; later replicates will be worked only after performance in the early batches is evaluated to the greatest extent possible and the number of replicates for subsequent release will be adjusted accordingly. Standard ABS and targeted list samples will be assigned to separate sequences of replicates in order to track how many of each type are released.

2. Procedures for the Collection of Information

Sampled addresses will be submitted to the vendor to match the phone numbers in preparation for computer-assisted telephone interviewing (CATI). The match rate was about 60% in the previous surveys. Eligibility will be determined through a screening interview (see Attachment 2a). The project advance letter will be mailed to the addresses that are matched to a telephone number, and then the household will be contacted by phone to complete the telephone survey (Household Member Interview, Attachment 2b).

If respondents do not complete the CATI interview after eligibility has been established, if respondents have been unreachable during CATI (i.e., the household remains unscreened in CATI), or if the sampled addresses cannot be matched with phone number, they will be mailed a REACH Study Booklet (Attachment 2c), which is a Self-Administered Questionnaire (SAQ), with a $5 incentive. The individual SAQs will be returned within a postage-paid business reply envelope that will be provided to the household.

We will conduct follow-up face-to-face computer assisted personal interviews (CAPI) in a subsample of non-respondents in the Haitian community in Boston. The survey in the Eastern Band of Cherokee community in North Carolina will conducted entirely through CAPI. The sampling frame in this community is the 911 emergency list of addresses of American Indians.

To be eligible for the REACH US Risk Factor Survey, a person must be 18 years of age or older, live in a sampled household, and be in the targeted population for the community. NORC will collect this information about each member of the sampled households in the Screening Instrument. This information will then be used to determine member eligibility.

When collecting data via telephone or in person, all eligible persons within each sampled household will have an equal chance of selection. Up to two adults will be selected at random in communities that do not require age or gender oversampling. In those communities that need oversampling, we will take all eligible adults who meet the oversampling requirements (e.g., all women aged 40-64 years or all adults aged 65 years and older) plus up to two additional other adults (i.e., adults who did not meet the gender/age oversample requirements but do meet the race/ethnicity screening requirements for a community). For those cases with data collected via mail surveys, completed interviews will be requested from all adult household members, and eligibility determination and the selection algorithm will take place after data collection. Because of the complexities of the screening algorithm, it is neither practical nor prudent to ask a household respondent to attempt to determine eligibility and select household members. Instead, by collecting data from all household members in all mailed households, we will be able to determine eligibility post hoc and then apply the random selection algorithm to the respondents.

Our goal is to complete 900 interviews with adults in each community. The cumulative sample size is 2,700 persons from each community after 3 years and 4,500 after 5 years. Six communities focus on breast and cervical cancer, and more than 10 communities focus on cardiovascular disease and diabetes, respectively. The following table provides sample size and power calculations to detect statistically significant intervention effects. Data from previous years show that the intervention effect was between 0.1 through 0.4. Hence, with a total of 6 communities focusing on breast and cervical cancer, a cluster correlation of 0.01 and an intervention effect of 0.2, a survey of 900 persons per community would provide close to 84% power to detect statistically significant results in the third year. The power is above 95% for the health priority areas of cardiovascular disease and diabetes. In the fifth year, the powers will be even larger.

Sample Size and Power to Detect Statistically Significant Intervention Effects

Number of Communities	Sample size	Cluster correlation	Intervention Effect	Power
2	500	0.01	0.2	0.19
2	1000	0.01	0.2	0.20
2	3000	0.01	0.2	0.21
4	500	0.01	0.2	0.58
4	1000	0.01	0.2	0.62
4	3000	0.01	0.2	0.64
5	500	0.01	0.2	0.72
5	1000	0.01	0.2	0.75
5	3000	0.01	0.2	0.78
6	500	0.01	0.2	0.81
6	1000	0.01	0.2	0.84
6	3000	0.01	0.2	0.87
8	500	0.01	0.2	0.94
8	1000	0.01	0.2	0.95
8	3000	0.01	0.2	0.95
10	500	0.01	0.2	0.97
10	1000	0.01	0.2	0.98
10	3000	0.01	0.2	0.99

We will use the software SUDAAN to perform data analysis to account for the complex sampling design, clustering, and probability of selection. Analyses will be performed by health priority area or by race/ethnicity. When a community is intervening in a specific health priority area, the remainder communities are considered control communities. In addition, state-based data from the BRFSS will be used as comparison for the REACH U.S. intervention communities, since the questions used in both studies are the same.

3. Methods to Maximize Response Rates and Deal with Nonresponse

To increase the response rate, advance letters describing the survey are sent to sampled households. The letter is in English as well as in Spanish, Vietnamese, Chinese, Korean, Haitian Creole or Khmer depending on the surveyed community and targeted minority population. Trained and certified interviewers who speak the language of the respondent conduct telephone and in-person interviews. Multiple attempts are made over approximately a one-month period to contact the sampled household. These attempts are made at different times of the day and on different days of the week, including weekdays and weekend days.

If an interviewer encounters a refusal, the interviewing supervisor and Project Manager will determine whether to re-contact the respondent. Respondents who are extremely hostile or verbally abusive will not be re-contacted. All others will be considered for conversion. A refusal conversion attempt will be made on these refusals by a more experienced interviewer who is a refusal conversion specialist. After the second CATI refusal, a refusal conversion letter that addresses the respondents concerns will be mailed to the eligible CATI respondents. The advance letter and the follow-up letter are included in Attachment 2d. Some contents of the letter may be revised to address a specific concern that was stated by the household.

We plan to include $5 token of appreciation payment with the SAQ study booklet. The household level response rate was 25% to 30% over the previous rounds of surveys.

The interviewers for person-to-person interviews will be trained, locally-based community members who have knowledge of their communities.

In Attachment 8 (Non-Response Plan), we describe how we monitor unit nonresponse pattern, analyze unit nonresponse bias, and calculate unit response rate. Nonresponse patterns will be analyzed separately for each community. An assessment will be made for each community in terms of the mean square error of some key estimates. The impact of the variance increase that would be introduced by the additional variability in the weights will be compared to the impact of the component of nonresponse bias that would be eliminated by using the weights.

4. Tests of Procedures or Methods to be Undertaken

The current incentive plan is based on an experiment conducted in the first year of the REACH US evaluation data collection (see Attachment 4).

The questions used in this survey have been used in BRFSS, a state-based telephone survey, as well as in our previous REACH 2010 evaluation. The telephone interview procedures for the collection of information in this application have been used in the past years in the REACH 2010 and REACH U.S. evaluations. Languages other than English (Spanish, Vietnamese, Khmer, and Mandarin Chinese, Korean, and Haitian Creole) have been used previously.

NORC has been at the forefront of development and use of ABS. The following are examples of projects that employed an ABS approach: The Neighborhood Organization, Aging, and Health (NOAH), the Chicagoland Catholics Project, and Survey of Consumer Finance (SCF). NORC’s Senior Technical Advisor for Sampling, Colm O’Muircheartaigh, is an acknowledged leader and pioneer in using ABS and has written numerous technical papers. CDC has also conducted studies comparing ABS and RDD surveys. Michael Link, who served as CDC lead technical expert in the design, conduct and analysis of methodological studies related to ABS for BRFSS, is currently the Senior Advisor for our survey.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Colm O’Muircheartaigh, Ph.D.

Senior Fellow and Professor

University of Chicago Harris School

1155 E 60^th Street

Chicago, IL 60637

(773) 256-6000

colm@norc.uchicago.edu

Michael Link, Ph.D.

Chief Methodologist

The Nielsen Media Research

New York, NY

Michael.link@nielsen.com

Contactor collecting the data:

National Opinion Research Center (NORC)

55 E. Monroe Street, Suite 2000

Chicago, IL 60603

(312) 759-4000

7