Section B

TABLE OF CONTENTS

SUPPORTING STATEMENT

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe the potential respondent universe and any sampling or other respondent

selection method to be used

2. Describe the procedures for the collection of information

3. Describe methods to maximize response rates and to deal with issues of

non-response

4. Describe any tests of procedures or methods to be undertaken

5. Provide the name and telephone number of individuals consulted on statistical

aspects of the design

B. Collections of Information Employing Statistical Methods

The proposed study will employ statistical methods to analyze the information collected from respondents and draw inferences from the sample to the target population. It will use an overlapping dual frame design with respondents interviewed on landline phones and on cell phones. The following sections describe the procedures for respondent sampling and data tabulation.

B.1. Describe the potential respondent universe and any sampling or other respondent selection method to be used.

1. Respondent universe

The respondent universe theoretically consists of all persons aged 16 or older residing in households. However, since the survey will be administered by telephone, the sampling universe will be persons aged 16 or older living in households with working telephones. Also, since interviews will be conducted in only English and Spanish, any person who does not speak one of these two languages will be excluded from this study. Based on the prevalence of the use of English and Spanish in the United States, excluding respondents who speak a language other than English and Spanish will have minimal impact on the study. This study will sample from all telephone households in the United States, including Alaska, Hawaii, and the District of Columbia.

It is not anticipated that the use of a telephone household sample will cause coverage bias. Research has shown that telephone service for many households is intermittent, and that gains or losses in service are associated with changes in their financial situation.¹ By implication, telephone coverage is not a static phenomenon, but rather, the telephone population at any given point includes households that were recently part of the non-telephone population. Households with intermittent telephone service are similar on a variety of important demographic characteristics to households who usually lack telephones.^1,2 Thus, weighting using information on interruptions in telephone service can ensure the correct coverage in the final sample of persons including those in non-telephone households.

2. Statistical sampling methods

A national sample of 6,000 people will be interviewed regarding distracted driving behavior, attitudes, and knowledge. The survey will employ a partially overlapping dual frame design with national probability samples drawn from independent sampling frames for landline phones and for cell phones (with screening for cell-only and cell-mostly households). Both the landline and cell phone samples will be geographically stratified by NHTSA Region. The precision of sampling estimates are generally improved by stratification. Stratification involves the division of the total population into population subsets in which the homogeneity of elements within the groups is greater than the homogeneity of elements within the population as a whole. Sampling is conducted separately within each stratum. Since the amount of improvement in sampling precision as a result of stratification is a function of the degree of homogeneity achieved within strata, national area probability samples are generally stratified by region (since regional differences in population characteristics and lifestyles may be expected to introduce variability into national samples).

NHTSA segments the country into ten Regions for programmatic outreach, divided as follows:

Table 1. NHTSA Regions
Region	States
Region 1-New England Region	Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, Vermont
Region 2- Eastern Region	New York, New Jersey, Pennsylvania
Region 3- Mid Atlantic Region	Delaware, District of Columbia, Maryland, Virginia, West Virginia, North Carolina, Kentucky
Region 4- Southeast Region	Alabama, Florida, Georgia, South Carolina, Tennessee
Region 5- Great Lakes Region	Illinois, Indiana, Michigan, Minnesota, Ohio, Wisconsin
Region 6- South Central Region	Louisiana, New Mexico, Oklahoma, Texas, Mississippi
Region 7- Central Region	Iowa, Kansas, Missouri, Nebraska. Arkansas
Region 8- Rocky Mountain Region	Colorado, North Dakota, South Dakota, Utah, Wyoming, Nevada
Region 9- Western Region	Arizona, California, Hawaii
Region 10- Northwest Region	Alaska, Idaho, Oregon, Washington, Montana

The estimated distribution of the landline telephone population by stratum is calculated on the basis of data from the National Health Interview Survey (NHIS). State estimates are available from the NHIS, which will be used to calculate the Regional distributions. NHTSA will use the most recent NHIS estimates available at the commencement of the field period. Based on NHIS estimates of the geographic distribution of the target population, the landline sample will be proportionately allocated by stratum. Geographic location for the landline sample is based on the dominant zip code for listed numbers within selected telephone banks (see next paragraph for description of phone banks). The estimated distribution of the cell phone only and cell phone mostly population by stratum is also calculated on the basis of data from the National Health Interview Survey. Since there are no listed numbers for cell phones, the geographic location assigned to cell phone banks is based on its billing center. This leads to a significant misclassification of the location of cell phone households at the community level, but very little at the State level since billing centers are located in the same State as the household being served.

The landline sample will be drawn from telephone banks randomly selected from an enumeration of the Working Residential Hundred Blocks within the active telephone exchanges within the Region. The Working Hundreds Blocks are defined as each block of 100 potential telephone numbers within an exchange that includes one or more residential listings (i.e., this will be a list-assisted sample). A two-digit number will then be randomly generated for each selected Working Residential Hundred Block to complete the phone number to be called. By randomly generating these numbers, a process known as random digit dialing (RDD), every number in the sampling frame of Hundreds Blocks has an equal probability of selection regardless of whether it is listed or unlisted.

The RDD sample of telephone numbers is dialed to determine which are currently working residential household telephone numbers. Non-working numbers and non-residential numbers are immediately replaced by other RDD numbers selected within the same stratum in the same fashion as the initial number. Ineligible households are also replaced. Non-answering numbers are not replaced until the research protocol is exceeded. The systematic dialing of those numbers to obtain a residential contact should yield a probability sample of landline telephone numbers.

The next step is selecting the household member to interview. Once a household is contacted, the interviewer will introduce him/herself and the survey, and ask how many persons residing in the household are 16 or older. If there is only one person in that age range residing in the household, then the interviewer will seek to interview that person. If there are multiple people in that age range residing in the household, the interviewer will randomly select one of those household members by asking to speak with the age-eligible household member who had the most recent birthday. Only that randomly selected person will be eligible to participate in the survey; no substitution by other household members will occur. If the selected respondent is not available, then the interviewer will arrange a callback. In cases where no one residing within the household is in the eligible age range, the interviewer will thank the individual who answered the phone, terminate the call, and the number will be removed from the sample.

Although list-assisted landline RDD sampling provides only a small coverage error for landline telephone households within landline banks, the restriction of the sampling frame to only landline banks would introduce a much more serious coverage error in general population surveys. The increasing percentage of households that have abandoned their landline telephones for cell phones has significantly eroded the population coverage provided by landline-based surveys. The key group that is missing from landline RDD samples is the cell phone-only group. But there is also potential bias in landline samples from under-coverage of people living in households with landlines but who have a low likelihood of being reached on them because they rely on wireless telephones for all or almost all of their calls. The cell phone sample for this survey will therefore be composed of both cell phone only and cell phone mostly respondents.

Due to the higher cost of cell phone interviews compared to landline interviews, dual frame surveys typically allocate less sample to the cell frame, than is needed to meet proportional allocation, in order to keep costs down without sacrificing data quality. For the 2014 NSDDAB, proportions similar to the 2012 NSDDAB will be used given the cost differential between landline and cell phone interviewing. The 2012 NSDDAB used an overall proportion of 35.5% of the total sample coming from the cell phone frame, which proved to be more than enough in order to balance the weights for a dual frame sample. The most recent data published from the National Health Interview Survey³ shows 36.5% of adults being in cell phone-only households and 18.1% being in cell phone-mostly households during the latter half of 2012. The ratio between the two wireless phone user categories is 67% cell phone only and 33% cell phone mostly. Applying that ratio to the 2,128 interviews conducted with the cell frame results in 1,426 cell phone-only interviews and 702 cell phone-mostly interviews. Cell phone households not falling in either of these categories will be screened out of the cell phone sample.

The cell phone sample will be randomly selected from 1,000 banks used exclusively for cell phones, using RDD. Procedures for sample selection will be similar to those used in selecting the landline sample, except that the sample will not be list-assisted as it is in the landline sample. In addition, the cell phone will be treated as a single user device. This means the cell phone sample will not require the procedures used with the landline sample to select from multiple eligible household members. Upon contact and introduction, the interviewer will immediately ask questions to determine that the person on the phone is not in a situation that could pose a safety risk to that individual (e.g., driving at the time of the call). If the contacted individual is found to be in a situation that could pose a risk, the interviewer will terminate the call and call back another time. If it is safe for the contacted individual to proceed with the call, then the interviewer will move on to asking screening questions. These will include questions to categorize the contacted individuals as cell phone only or cell phone mostly. Cell phone users not falling into one of these two categories will be screened out as the cell phone sample is to be composed only of those two groups.

3. Over-sampling Persons Age 16-34 Years

Data from the 2012 NSDDAB showed younger adult age groups are more likely to engage in distracted driving activities when compared to their older peers,⁴ making them a group from whom it is especially important to collect information for this survey. However, the low response rates typically achieved with young adults will result in their composing a far smaller proportion of the achieved sample compared to their actual prevalence in the 16 and older population if left to simple random selection procedures. The reduced sample combined with lower response rates among the age segments where the behavior of interest is more prevalent would lead to sub-optimal numbers of younger drivers, which are of particular interest to NHTSA responding to the NSDDAB. This survey therefore will engage in modest over-sampling of respondents 16 through 34.

Table 2 shows the planned distribution of respondents across the landline sample, the cell phone sample, and a fixed landline over-sample numbering 500 respondents. The expected size of the 16 through 34 age group in the landline and cell phone samples is derived from figures obtained from the 2012 NSDDAB, which used methods similar to those that will be used for the 2014 National Survey on Distracted Driving Attitudes and Behavior. Adding a 500 interview over-sample will raise the number of people 16 through 34 participating in the survey to a projected 1,738 respondents (29% of the sample). The over-sample will be conducted exclusively on landline phones using an independent RDD sample. The landline frame will be the frame with the lower proportion of 16-34 year olds (as compared to the cell frame) and there is a need to increase the proportion within this frame. Also, considerable cost savings will be realized by limiting the over-sample to the landline frame. However, the Table shows that the majority of interviews conducted with young adults will still be obtained from the cell phone frame (n=918).

Table 2. Distribution of Sample (2014 NSDDAB)
Sample	Total n	Total %	16-34 n	16-34 %
Landline	3,372	56.2%	320	18.4%
Cell Phone	2,128	35.5%	918	52.8%
Over-sample	500	8.3%	500	28.8%
Total	6,000	100%	1,738	100%

Sampling methods applied to the over-sample will be the same as those applied to the cross-sectional landline sample. The difference will be that the age screening will be for ages 16 through 34 instead of 16 and older. The allocation of sample by region for the young adult sample would be proportionate to the regional distribution of the age 16 through 34 landline telephone population.

4. Precision of sample estimates

The objective of the sampling procedures described above is to produce a random sample of the target population. This means that with a randomly drawn sample, one can make inferences about population characteristics within certain specified limits of certainty and sampling variability.

The margin of error, d, of the sample estimate of a population proportion, P, equals:

Where t_α equals 1.96 for 1-α = 0.95, and the standard error of P equals:

Where:

deff = design effect arising from the combined impact of the random selection of one eligible individual from a sample household, and unequal weights from other aspects of the sample design and weighting methodology, and

n = the size of the sample (i.e., number of interviews)

Using these formulas, the margin of error for a sample size of 6,000 interviews is d = 0.015 to 0.018, using an average deff of 1.5 to 2.0 and setting P equal to 0.50. The Pew Research Center, the Kaiser Family Foundation, and AP/GfK report average design effects for their overlapping dual frame sample designs of 1.39, 1.33 and 1.68, respectively⁵, making it reasonable to assume that the design effect for the 2014 NSDDAB will be around 1.5 to 2.0.

An important part of the analyses will be comparison of the 2014 data with the data from the previous administrations. The minimum detectable effects (with 80% power, alpha = .05) for key comparisons between the 2014, 2012, and 2010 surveys is 2.2% given that the sample sizes are roughly 6,000 each year (See Figure 1).

Figure 1. Power Curve for 2 samples of n=6,000⁶

4. Sample weighting – Landline sample

Starting with the calculation of the weights for the landline RDD sample, the base

sampling weight will be equal to the reciprocal of the probability of selecting the household’s telephone number in its geographic sampling stratum. Thus, the base sampling weight will be equal to the ratio of the number of telephone numbers in the 1+ working banks for that geographic stratum to the number of telephone numbers drawn from those banks and actually released for use. A sampling weight adjustment will then be implemented to compensate for over-sampling of households containing respondents aged 16 through 34 in the independent landline RDD sample. Next, the reciprocal of the number of voice-use landline telephone numbers in the household (up to a maximum of three such numbers) will be calculated. It will compensate for a household’s higher probability of selection when it has multiple voice-use residential telephone lines. As noted earlier we will compensate for the exclusion of non-telephone households by using an interruption in landline telephone service adjustment. A within-household weighting factor will then be created to account for the random selection of one respondent from the household. For the independent over-sample the within household weight factor is the number of persons age 16-34 years in the household. The design weight then equals the product of each of the weight components described above.

Adjustment for unit non-response will be based on at least two cells in each of the regions. The non-response adjustment cells will be defined by whether the respondent (or person who was selected for an interview but did not complete it) resides in a directory-listed number household. Other telephone exchange variables may be considered in the development of the unit non-response adjustments. The non-response adjusted weight equals the design weight times the non-response adjustment factor. For example, the non-response adjustment for the directory listed sample is:

where

= the non-response adjusted weight for the i-it respondent with a directory listed telephone number,

= the design weight for the i-th respondent with a directory listed telephone number,

= sum of the base sampling weights for respondents with a directory listed telephone number, and

= sum of the base sampling weights for non-respondents with a directory listed telephone number.

4. Sample weighting – Cell phone sample

For the cell phone sample the first step is to assign a base sampling weight that equals the reciprocal of the probability of selection of the cellular telephone number. The base sampling weight will be calculated separately for each NHTSA region. The cellular phone will be treated as a personal device so there will be no random selection of a respondent. The design weights will then be adjusted for unit non-response separately for cell-only respondents and for cell-mostly respondents. At this point we have non-response adjusted weights for the cell phone sample.

4. Sample weighting – Combined sample

The first step involves compositing the cell-mostly respondents from the landline sample with the cell-mostly respondents from the cell phone sample. The compositing factor, λ, will be calculated using the effective sample sizes for the two cell-mostly samples. In the final file used for weighting, all contacted households from the landline cross-section sample, the cell phone sample, and the landline over-sample will be included. It is important to include all contacted households, rather than only completed interviews since the contacted households provide useful information for weighting purposes.

The population control totals for the raking of the combined sample will be based primarily on the latest Census Bureau population estimates (age by gender by race/ethnicity) and the American Community Survey (ACS) to develop some of the population control totals for socio-demographic variables such as education. Battaglia et al. (2008)⁷ have shown the potential for strong bias reduction in RDD samples from the inclusion of socio-demographic variables in the raking. We will also use the results of the non-response bias analysis to inform which variables should be included in the raking.

First we identify the margins that will be used in raking: Region, Age x gender, race/ethnicity, and telephone usage. The control total for each category of each margin is established. The raking algorithm is iterative. In the first iteration, the weights are adjusted such that sum of the weights agree with the control totals in different categories of the first margin. Then these weights are adjusted such that the sum of the weights agrees with the control totals of the second margin. The first iteration process is continued till the weights are adjusted to agree with the control totals of the last margin. The sum of these weights of course will not agree with the control totals of other margins. At the end of the first iteration we will have a set of weights different from the original nonresponse-adjusted sampling weights. In the second iteration, the whole process is repeated. These iterations continue until the weights converge, meaning the sum of the weights agree or differ very little from the controls totals in each margin of each category. It is possible to set up the desired difference between control totals in the various margins and the sum of the raked weights to declare that the iterative process has converged.

One might rake the combined sample to the socio-demographic control totals discussed above, but it is important to also have raking control totals related to telephone usage: number of persons age 16 years and older that only have landline telephone service, number of persons age 16 and older that only have cellular telephone service, and number of persons age 16 years and older that have dual (landline and cellular) telephone service, with this last group split in cell-mostly and not cell-mostly subgroups. Such estimates can be obtained for the U.S. from the National Health Interview Survey (NHIS).

The sampling weight assigned to a respondent will be based on probability of selection in the NHTSA Region that includes the billing address at the time of selection.  The sampling weight is the inverse of the probability of selection of the respondent.  After data collection, if the respondent indicates a ZIP code which is in a different NHTSA Region, the respondent will be assigned to the new region but will maintain the original sampling weight.  The assumption is that he or she is representing other persons who changed regions between the time of selection and data collection in the non-sampled part of the universe. Post-stratification adjustment will be made to this weight along with the weights of those persons who did not change region using the control total for the new region.  One result of such changes is a slight increase in the variance of the estimates because of different weights within a stratum especially if we are using equal probability sampling for selection of persons within a stratum.

To develop the final weights, the project statistician will rake the non-response adjusted weights, iterating to get close agreement on the margins for NHTSA region, gender by age, race/ethnicity, and telephone usage group. The latest version of the Izrael, Battaglia, Frankel (IBF) SAS raking macro⁸ will be used to implement the raking. The project staff will also examine whether any additional margins should be included in the raking and if weight trimming is appropriate. Statistics related to the distribution of the weights after raking, such as the coefficient of variation, will be examined to determine if weight trimming should be used to reduce variability in the final weights. The IBF SAS raking macro allows for weight trimming along four dimensions:

1. Global low weight cap value factor: Mean raking input weight times A.

2. Global high weight cap value factor: Mean raking input weight times B.

3. Individual low weight cap value (ILCV) factor: Respondent's weight times C.

4. Individual high weight cap value (IHCV) factor: Respondent's weight times D.

For the global factor, the raking input weights are ratio-adjusted to add to the population size. Then the mean raking input weight is calculated. During the raking process the weights are not allowed to exceed the global high weight cap value which for this survey is the mean weight times 10. The individual weight cap factor is simply the raking input weight for a respondent times the value given. For example, during the raking iterations, the weight of a respondent is not allowed to exceed the value chosen (individual weights times the factor).

The IBF SAS raking macro carries out the weight trimming based on the four weight trimming criteria during the raking iterations. Assuming convergence is achieved, the weights are therefore trimmed and the weighted sample is in very close agreement with the population control margins.

Based on the 2012 NSDDAB, our initial plan for weight trimming using the IBF SAS raking macro and recommended factor values is as follows:

1. Global low weight cap value factor: Mean raking input weight times 0.1.

2. Global high weight cap value factor: Mean raking input weight times 10.0.

3. Individual low weight cap value (ILCV) factor: Respondent's weight times 0.2.

4. Individual high weight cap value (IHCV) factor: Respondent's weight times 5.0.

4. Variance estimation

Variance estimation will need to rely on statistical software such as SUDAAN, however, existing software will not directly calculate standard errors for overlapping dual frame designs. Rather, it will be necessary to calculate the variance for each of the two samples and use the results to calculate the variance for the combined sample.

The dual frame composite estimator⁹ commonly used for overlapping dual frame telephone samples is:

is the composite (also referred to as the average) estimator. LL designates the landline sample and CELL designates the cellular sample. LL-O, CELL-O, and DUAL identify the landline-only, cell-only, and dual service domains, respectively. Thus, _{ } and _{ } are the estimates from the two non-overlapping domains. and are the estimates for the overlap domain from the landline and cell samples, respectively. λ is the compositing factor for the overlap component ( ).

For our survey there is a partial overlap. The DUAL domain is defined as dual users who are cell mostly. The non-overlap domain in the landline sample includes dual users who are not cell mostly in addition to landline only.

Most dual frame telephone surveys conducted today use the composite estimator and treat λ as fixed.¹⁰ If we treat λ as fixed then the variance of can be written as:

_{The first component will be estimated for the landline sample using SUDAAN by taking the complex nature of the sample design into account. The second component will be estimated for the cellular sample also using SUDAAN by taking the complex nature of the sample design into account.}

B.2. Describe the procedures for the collection of information.

1. Call strategy

The National Survey on Distracted Driving Attitudes and Behavior (NSDDAB) will be administered to a randomly selected national sample of 6,000 respondents age 16 and older. Approximately 3,372 interviews will be conducted with a landline cross-sectional sample, 2,128 interviews conducted with a cell phone cross-sectional sample, and 500 interviews conducted with a landline over-sample of individuals age 16 through 34.

Interviewing will be conducted according to a schedule designed to facilitate successful contact with sampled households and complete interviews with the designated respondent within those households. Initial telephone contact will be attempted during the hours of the day and days of the week that have the greatest probability of respondent contact. This means that much of the interviewing will be conducted between 5:30 p.m. and 9:30 p.m. on weekdays; between 9:00 a.m. and 9:30 p.m. on Saturdays; and between 12:00 noon and 9:30 p.m. on Sundays. Since interviewing will be conducted across several time zones, the interviewing shift lasts until 12:30 a.m. at night. Since the interviews are expected to average about twenty minutes in length, the last contact attempt will not be made later than 9:00 p.m. according to local time unless scheduled by the respondent.

The 2014 NSDDAB will include daytime calling within its calling algorithm. Research has demonstrated that a variation in calling times can reach subsamples disproportionately. For the general population, evenings have the highest contact rate as opposed to weekdays and weekends. But for low income groups and minorities, daytime calls increase the probability of contacting the respondent. Lower income respondents are more likely to have irregular work hours (e.g. shift work, etc.) and sometimes have lower labor market participation, which places them at home during the day.

The NSDDAB will employ a 10-20 call strategy for landline numbers where up to 10 call attempts will be made to ringing but unanswered numbers before the number is classified as a permanent no answer and up to 20 calls if contact is made with someone in the household. Callbacks to unanswered numbers will be made on different days over a number of weeks according to a standard callback strategy. If contact is made but the interview cannot be conducted at that time, the interviewer will reschedule the interview at a time convenient to the respondent. If someone picks up the phone, but terminates the call before in-house selection of a designated respondent can be made, then the Contractor will apply a protocol to maximize participation while minimizing respondent annoyance. Fewer calls will be made to cell phone numbers. The calling strategy for cell phones will be 5 call attempts made to ringing but unanswered numbers before the number is classified as a permanent no answer and up to 10 calls if contact is made with the respondent.

2. Protocol for answering machines and voice mail

When contact is made with an answering machine or voice mail, a message will be left according to a set protocol. For landline numbers, a message will be left on the 5^th, 7^th, and 9^th attempts to contact a household member. The message will explain that the household had been selected as part of a national USDOT study, ask that the Contractor’s toll-free number be called to schedule an interview, and include reference to the NHTSA website which will include information about the survey so that prospective respondents can verify the survey’s legitimacy. For cell phone numbers, the message will be left on the 1^st attempt so that the cell phone user can quickly attach an identity to the number s/he is seeing on the phone.

3. Initiating the interview

When a household is reached in the landline sample, the interviewer will screen for age eligibility. If only one household member is age eligible, then the interviewer will seek to interview that individual. If there is more than one eligible household member, then the interviewer will randomly select one respondent from among them and seek to interview that person. Appointments will be set up with respondents if it is inconvenient for them to be interviewed at the time of contact. If the randomly selected respondent is not available at the time of contact, then the interviewer will ask what would be a good time to call back to reach that person.

For the cell phone sample, respondents will first be asked if they are in a situation where it would be unsafe to speak with the interviewer. If the respondent says “Yes,” then the interviewer will say that he or she will call back at another time, and immediately terminate the call. Once a cell phone user is reached at a safe time, the interviewer will first screen for age eligibility. If the cell phone user is eligible to participate, then the interviewer will seek to proceed with the interviewer. If it is an inconvenient time for the respondent, then the interviewer will try to set up an appointment.

4. Spanish language interviews

If a non-bilingual interviewer encounters a language barrier at the telephone number, either with the person answering the phone or with the designated respondent, the interviewer will thank the person and terminate the call. If a case is designated as Spanish language, it will be turned over to the next available Spanish-speaking interviewer. The Spanish-speaking interviewer will utilize a Spanish version of the questionnaire. The translation of the questionnaire from English to Spanish will have been checked prior to the survey field period using back translation methods.

5. Interviewer monitoring

Each interviewer will be monitored throughout the course of the project. The Supervisor will evaluate the interviewer on his or her performance and discuss any problems that an interviewer is having with the Field Director. Before the end of the interview shift, the Shift Supervisor will discuss the evaluation with the interviewer. If the interviewer cannot meet the Contractor’s standards, he or she will be removed from the project.

All interviewers on the project will undergo two types of monitoring. The Supervisor will sit at a computer where s/he can see what the interviewer has recorded, while also audio-monitoring the interview. The audio-monitoring allows the Supervisor to determine the quality of the interviewer's performance in terms of:

1. Initial contact and recruitment procedures;

2) Reading the questions, fully and completely, as written;

3) Reading response categories, fully and completely, (or not reading them) according to study specifications;

4) Whether or not open-ended questions are properly probed;

5) Whether or not ambiguous or confused responses are clarified;

6) How well questions from the respondent are handled without alienating the respondent;

7) Avoiding bias by either comments or vocal inflection;

8) Ability to persuade wavering, disinterested, or hostile respondents to continue the interview; and,

9) General professional conduct throughout the interview.

The Supervisor will also monitor the interviewer's recording of survey responses as the Supervisor's screen emulates the interviewer's screen. Consequently, the Supervisor can see whether the interviewer enters the correct code, number, or verbatim response.

B.3. Describe methods to maximize response rates and to deal with issues of non-response.

1. Cognitive testing

The 2010 questionnaire was cognitive tested in order to identify any problem issues that could deter respondents from completing the survey. Based on the cognitive testing results, a number of questions were revised or deleted. In 2012, the Spanish version of the survey was added. The Spanish version was cognitive tested with nine Spanish speaking people in order to identify any problem issues that could deter respondents from completing the survey.

2. Initial contact

The initial contact with the designated respondent is crucial to the success of the project. Most refusals take place before the interviewer has completed the survey introduction (usually within the first 30 seconds of the call). Numerous studies have shown that an interviewer's manner of approach at the time of the first contact is the single most important factor in convincing a respondent to participate in a survey. Many respondents react more to the interviewer and the rapport that is established between them than to the subject of the interview or the questions asked. This positive first impression of the interviewer is key to securing the interview.

While the brief introduction to the study concerning its sponsorship, purpose, and conditions are sufficient for many respondents, others will have questions and concerns that the interviewer must address. A respondent's questions should be answered clearly and simply. The interviewers for the NSDDAB will be trained on how to answer the most likely questions for this survey. A hard copy of Frequently Asked Questions will be provided to each interviewer at their station. The interviewers also will be trained to answer all questions in an open, positive, and confident manner, so that respondents are convinced of the value and legitimacy of the study. If respondents appear reluctant or uncertain, the interviewer will provide them with a toll-free number to call and the NHTSA website address to verify the authenticity of the survey. Above all, the interviewer will attempt to create a rapport with the study subject and anyone in the household contacted in the process of securing the interview.

3. Refusal documentation and tracking

Higher response rates can be achieved through procedures built on careful documentation of refusal cases. Whether the initial refusal is made by the selected respondent, a third party, or an unidentified voice at the end of the phone, the CATI system used for this survey will be programmed to move the interviewer to a non-interview report section. The interviewer will ask the person refusing to tell him/her the main reason that he or she doesn’t want to do the interview. The interviewer will enter the answer, if any, and any details related to the refusal into the program to assist in understanding the strength and the reason for the refusal. This information, which will be entered verbatim in a specific field in the CATI program, will be part of the formal survey reporting record, and will be reviewed on a daily basis by the Field Manager and, on a weekly basis, by the Project Director. Based on the pattern and distribution of refusals, a strategy for refusal conversion will be formalized.

The Survey Manager and the Project Director will review the information about refusals and terminations on the CATI system on an ongoing basis to identify any problems with the contact script, questionnaire, or interviewing procedures that might contribute to non-participation. For example, they will scrutinize the distribution of refusals by time of contact, as well as any comments made by the respondent, in order to determine whether calling too early or too late in the day is contributing to non-participation. Also, the refusal rate by interviewer will be closely monitored.

In addition to relying on the CATI data records, the Project Director and Project Manager will also consult with the interviewing Shift Supervisor, who has monitored the interviewing and debriefed the interviewers. The information from these multiple sources provides solid documentation of the nature and sources of non-response. It will also help to generate the "conversion script” which will be developed by the Project Director.

4. Refusal conversion

Prospective respondents that refuse to participate in the survey will be re-contacted approximately one to two weeks after the initial refusal. These initial refusals will be asked to reconsider and participate in the survey. People identified as “hard” or “not to be converted” refusals will not be included in this refusal conversion effort.

The process of converting terminations and refusals, once they have occurred, will involve the following steps. First, there will be a diagnostic period as described in the documentation and tracking section above. Second, after enough time has passed to see a large enough sample of refusals and terminations, the Project Director and his staff will work out a refusal conversion script. Third, the refusal conversion effort will be fielded with re-interview attempts scheduled about one-to-two weeks after the initial refusal (it is important to have a “cooling off” period before re-contacting the household). Conversions of interviews that are more than half complete would not be delayed this long. Fourth, the Project Director and Operations Manager will receive the outcomes of the refusal conversion efforts on a daily basis. Revisions of the script or the procedures would be made if indicated by the ongoing results of the conversion effort.

Past experience suggests that about 20% of initial refusals will reconsider. Success in refusal conversion tends to depend more heavily on the difficulty of the survey and the study population, i.e., the hardness of the initial refusals, than it does with the level of effort on refusal conversions. Nonetheless, sensitivity in the diagnosis of reasons for non-participation and creativity in responding to those problems can have a real effect on refusal conversion.

5. NHTSA website information

It is not uncommon for behavioral self-report surveys such as these to generate incoming calls or emails to the U.S. Department of Transportation from people who have been contacted by the survey. In a large percentage of the cases, it is an inquiry to check the survey’s legitimacy. People are wary about attempts to extract information from them, wondering if the caller is truthful in identifying the source of the survey and the use of their information.

To help allay these fears, NHTSA will place on its website information that prospective respondents can access to verify the survey’s legitimacy. The interviewers will provide concerned respondents with a NHTSA web address where they will find information on the source of the survey and why it is important for them to participate. A toll-free number to reach the survey Contractor will also be provided so they can schedule an interview.

6. Non-response bias analysis

A non-response bias study will be conducted for the 2014 NSDDAB. Non-response is the failure to obtain survey measures on all sampled individuals. There are two main potential consequences from non-response: (1) non-response bias and (2) underestimation or overestimation of the standard errors.¹¹ The main challenge posed by non-response is that without a separate study (i.e., a non-response bias analysis), one can never be sure of whether the non-response has introduced bias or affected the estimation of the standard errors. The mere presence of non-response, even high non-response, does not necessarily imply bias or non-response error.

The non-response bias study will entail two analyses. First, the demographic composition of the 2014 NSDDAB landline sample will be compared to the demographics of non-respondents as determined by auxiliary data providing demographic information by area code/prefix combination for the non-respondent phone numbers. The auxiliary data will be purchased from an external vendor and will represent the latest and most recent population information at the zip code level. The analysis will assess whether the landline non-respondents are demographically different from those in the sample and whether some of the estimates are correlated with the propensity to respond. If some of the estimates are correlated with the propensity to respond, there is a chance that there might be some bias in those estimates.

Additionally, there will be a non-response follow-up (NRFU) survey with a sub-sample (n=200) of households that failed to respond in either the landline or cell phone sample. The NRFU has three main objectives: 1) investigate the reasons for non‐response, 2) assess the risk of non-response error to model estimates, and 3) identify changes to the survey protocol that may help to reduce the non‐response rate and/or the risk of non‐response bias. Because these are people who either refused to participate in the main survey or otherwise avoided participation, there are limitations on how many respondents one can expect to participate in the NRFU. However, assuming that the sub-sample of 200 is representative of all non-responders, the margin of error of +/- 7 percentage points at the 95% confidence interval would be sufficient to assess differences between non-responders and a main survey sample. Based upon past NRFU studies which have been conducted for NHTSA, including the 2012 NSDDAB, the sample size of 200 households is large enough to estimate factors that contributed to non-response, assess the potential for non-response bias in key substantive variables, and determine key socio-demographic variables to include in the final post-stratification adjustments.

The NRFU features an abbreviated survey instrument in which the household is asked to provide a few basic socio-economic and demographic variables, reasons for non-participation, and responses to a few substantive survey items. NRFU households will each be offered $20 for their participation, as this amount has been effective in past non-response follow-up studies conducted by the Contractor. The results of the non‐response follow‐up survey will provide valuable information for assessing the extent to which non-response bias affects the overall survey respondent sample and for developing weighting adjustments that minimize the impact of non-response bias on sample statistics and model parameters.

The analysis will include comparisons of key variables from the NSDDAB respondents with the same variables for the NRFU respondents. This will provide insights about the direction and magnitude of possible non-response bias. The project team will investigate whether any differences remain after controlling for major weighting cells (e.g., within race and education groupings). If weighting variables eliminate any differences, this suggests that weighting adjustments will reduce non-response bias in the final survey estimates. If, however, the differences persist after controlling for weighting variables, then this would be evidence that the weighting may be less effective in reducing non-response bias.

B.4. Describe any tests of procedures or methods to be undertaken.

1. Testing the programming

The project team will test the CATI program thoroughly in test mode – running the interviewing program through multiple loops. The analytical staff will attempt to test all possible response categories for each question in order to identify embedded logic errors, as well as obvious skip problems. Several analysts may test the program simultaneously to identify problems quickly, and to double check the comprehensiveness of the testing.

2. Survey pretest

Pre-test interviews will be conducted with 30 (15 English speaking and 15 Spanish speaking) respondents prior to the formal administration of the survey in order to test all of the survey systems, the survey instrument, and the CATI programming. The pretest will be conducted under conditions identical to the planned conditions for the final administration of the survey. It will be used to evaluate the wording of questions and response categories in terms of clarity and confusion for respondents; assess the flow of the interview, including question sequencing and skip patterns; check the time required to administer the questionnaire to see if it differs from the original estimate; assess item non-response to see if there are problems with particular questions; ensure that a smooth interview occurs with previous information guiding later questions; ensure random rotation of specified questions; check that invalid responses are not entered; and review and evaluate the full range of procedures.

Immediately following the interviews, the interviewers will participate in a debriefing session to report any problems or observations they have regarding the instrument to the field manager. The field manager will consolidate the observations and suggestions along with his own recommendations. These will be returned to the project director along with the hard copy of the pretest interviews. Interview timings will be provided as well. The completed questionnaires will be analyzed, along with the extensive comments on the content and administrative aspects of the instruments made by the interviewers, monitors and research staff observing the pretest. Revisions to procedures, CATI programming, and the survey instrument will be made as appropriate.

B.5. Provide the name and telephone number of individuals consulted on statistical aspects of the design.

The following individuals have reviewed technical and statistical aspects of procedures that will be used to conduct the 2014 National Survey on Distracted Driving Attitudes and Behavior:

Paul Schroeder, MA

Vice President

Abt SRBI, Inc.

8405 Colesville Road, Suite 300

Silver Spring, MD 20910

(301) 628-5502

KP Srinath, PhD

Senior Statistician

Abt SRBI, Inc.

8405 Colesville Road, Suite 300

Silver Spring, MD 20910

(301) 628-5527

Kristie Johnson, PhD

Office of Research and Technology

DOT/National Highway Traffic Safety Administration

1200 New Jersey Ave, SE

Washington, DC 20590

(202) 366-2755

1 Keeter, S. (1995). Estimating non-coverage bias from a phone survey. Public Opinion Quarterly, 59, 196-217.

2 Frankel, M., Srinath, K.P., Hoaglin, D.C., Battaglia, M.P., Smith, P.J., Wright, R.A., & Khare, M. (2003). Adjustments for non-telephone bias in random-digit-dialing surveys. Statistics in Medicine, 22, 1611-1626.

3 Blumberg, S.J. & Luke, J.V. Wireless substitution: Early release of estimates from the National Health Interview Survey, July – December 2012. Retrieved from Centers for Disease Control and Prevention, National Center for Health Statistics website: http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201306.pdf

4 Schroeder, P., Meyers, M., & Kostyniuk, L. (2013, April). National survey on distracted driving

attitudes and behaviors – 2012 (Report No. DOT HS 811 729). Washington, DC: National

Highway Traffic Safety Administration. Retrieved from www.nhtsa.gov/staticfiles/nti/pdf/811729.pdf

5 Lambert, D., Langer, G., & McMenemy, M. (2010). Cell phone sampling: An alternative approach.

Paper presented at the 2010 AAPOR Conference, Chicago, IL. Retrieved from http://abcnews.go.com/images/PollingUnit/Cell-OnlySampling-Lambert-Langer-McMenemy-2010.pdf

6 Power Curve was created using G*Power 3.1.9. Parameters Entered: Power=80%, Alpha=.05, Sample Size₁=6,000, Sample Size₂=6,000, Proportion=.50. Program available from http://www.softpedia.com/get/Science-CAD/G-Power.shtml

7 Battaglia, M.P., Frankel, M.R. & Link, M.W. (2008). Improving standard poststratification techniques for random-digit-dialing telephone surveys. Survey Research Methods, 2(1), 11-19.

8 Izrael, D., Battaglia, M.P., & Frankel, M.R. (2009). Extreme survey weight adjustment as a component of sample balancing (a.k.a. raking). Paper 247-2009. Proceedings of the SAS Global Forum 2009 Conference, Cary, NC. Retrieved from http://support.sas.com/resources/papers/proceedings09/247-2009.pdf

9 Lohr, S.L. (2009). Multiple frame surveys. In D. Pfeffermann and C.R. Rao (Eds.), Handbook of statistics 29A. Sample surveys: Design, methods and applications (pp.71-88). Amsterdam: Elsevier.

10 Brick, J.M., Cervantes, I.F., Lee, S. & Norman, G. (2011). Nonsampling errors in dual frame telephone surveys. Survey Methodology, 37(1), 1-12. Retrieved from http://www.statcan.gc.ca/pub/12-001-x/2011001/article/11443-eng.pdf

11 The standard error for a survey with a low response rate will be larger than one with 100% response. However the statistical tests conducted with the data will assume a 100% response rate, hence the potential for under/over-estimating of the standard error.

20