Supporting Statements: Part B
Alcohol-Impaired Driving Segmentation Study
OMB Control Number: 2127-(New)
Abstract:1
The National Highway Traffic Safety Administration (NHTSA) is seeking approval of this new information collection request (ICR) to allow the agency to conduct a one-time voluntary study to obtain information to better understand attitudes and behaviors related to alcohol-impaired driving that will be used to enhance and refine communication strategy and tactics (i.e., more effectively target and message at-risk drivers and motorcycle riders). The study will survey drivers and motorcycle riders ages 21- to 54-years-old because this age range represents the greatest number of alcohol-related driving/riding fatalities according to NHTSA’s Fatality Analysis Reporting System (FARS).2
The research study will include two components, both being one-time collections. The first component will involve a series of online webcam interviews that will collect qualitative information that will be used to improve the quantitative survey that will be administered in the second component. The quantitative survey will be administered online and by phone (and potentially supplemented by mail if needed). After collecting the data, segmentation analysis will be done to classify drivers and motorcycle riders according to segments based on common demographics, drinking behaviors, attitudes about drinking and driving/motorcycle riding, and lifestyle characteristics. The segmentation profiles will be used by NHTSA’s Office of Communications and Consumer Information (OCCI) to better target and reach intended audiences with communications messages and techniques that are relevant and meaningful to people within the target market.
B. JUSTIFICATION
B.1 Describe the potential respondent universe and any sampling or other respondent selection to be used.
The respondent universe includes anyone who meets the following criteria:
Age 21- to 54-years-old
Currently possess a valid driver’s license for a motor vehicle and/or motorcycle
Drive, on average, 10 or more miles per week and/or ride a motorcycle, on average 10 or more miles per week
Consume alcoholic beverages multiple times per month
Meets at least one of the following criteria:
Heavy drinker (has more than 1.5 drinks per hour)
Drove vehicle after drinking, rode motorcycle after drinking or rode with a driver who had also been drinking in the last 12 months
Attitudes align with at least three out of seven risk statements used in screening in prior impaired driver segmentation research for NHTSA
Respondents are considered to align with each of the following risk statements if they say it does not at all or does not too well describe them:
I only use a designated driver if absolutely necessary
I know when I have had too much to drink to drive safely
It is better to be safe than sorry when it comes to driving after drinking
I would not get upset if a friend or family member tried to stop me from driving after I had had a few drinks
Respondents are considered to align with each of the following risk statements if they say it somewhat, very well or extremely well describes them:
It is hard to have a good time when I’m going out if I have to be the designated driver
I typically do not worry about how much alcohol I am drinking
I sometimes do not realize that I have had too much to drink
Has been stopped by law enforcement for reckless driving or driving while impaired
Has been involved in a crash in the last 12 months
*NOTE: Drivers/motorcycle riders ages 21- to 54-years-old constitute the greatest number of alcohol-related driving/riding fatalities according to NHTSA’s Fatality Analysis Reporting System (FARS)3.
The research will contain two phases—the first being a cognitive test using a series of in-depth webcam interviews, during which participants will complete the survey and debrief with a moderator during the process. The goal of the cognitive test is to ensure each question is clear, consistently understood and offers appropriate response options. The second phase will include a mixed-mode quantitative survey.
The qualification criteria will be identical for the two phases, but participants will be sourced differently. The cognitive testing will rely on a convenience sample and will source participants through proprietary databases maintained by recruiting partners. The quantitative phase will utilize the National Opinion Research Center’s (NORC) AmeriSpeak panel.
AmeriSpeak is the first U.S. multi-client household panel to combine the speed and cost-effectiveness of panel surveys with enhanced representativeness of the U.S. population, an industry-leading response rate and the NORC Card, an innovative sample quality report card. Since its founding by NORC at the University of Chicago in 2015, AmeriSpeak has produced more than 300 surveys, been cited by dozens of media outlets and become the primary survey partner of The Associated Press. AmeriSpeak’s sampling provides better representation than other panels for hard-to-reach populations, including low-income households, less-educated persons, young adults, rural households, persons who are less interested in the news, and social and political conservatives. AmeriSpeak promotes itself as the most scientifically rigorous multi-client panel available in the U.S. market.4
AmeriSpeak uses an invitation-only probability panel design that is representative of the US population. Because the panel is representative of the US population, survey results can be extrapolated to the population under study. The difference between probability and non-probability samples is the ability to confidently project results to the underlying population. While NORC’s AmeriSpeak sample is a panel, it is recruited and built using probability-based sampling techniques. Additionally, NORC will use other samples (including phone and mail if needed) to supplement their panel and access hard-to-reach populations.
B.2 Describe the procedures for the collection of information.
Once the survey is ready for testing, a recruiting partner will begin outreach to find nine English-speaking and nine Spanish-speaking respondents who meet the screening criteria and are available and interested in participating in the cognitive test. The cognitive test will be conducted using a series of in-depth webcam interviews during which a moderator guides the participant through the survey and debriefs them after each question to see if the wording is clear and easy to understand and if the available response options are adequate. The moderator will take notes and record the interview to create transcripts. Notes and transcripts will be summarized into a final report that presents findings in aggregate and highlights suggested changes to improve the survey for comprehension.
The survey program will be updated based on recommended changes from the qualitative phase, and the quantitative phase will begin. NORC will be engaged and will manage outreach using their AmeriSpeak probability panel. This will consist primarily of email invitations but will also be supplemented with outbound phone calls, mail surveys and intercept surveys as needed. The expectation is approximately 5%–10% of completes will be obtained by these supplemental methods, and most, if not all, of the supplemental completes will be collected with phone surveys. NORC presented a paper at the American Association for Public Opinion Research (AAPOR) on the trade-offs of using a mixed-mode design.5 The findings indicate that a mixed mode design improves sample coverage and representativeness of a survey but can introduce mode effects on questions where there is a potential social desirability bias. It is important to frame questions in a neutral manner and limit items prone to social desirability bias. Weighting can be applied to help minimize differences.
The survey will field until it reaches its target of 5,400 completes. Approximately 20,930 respondents will be screened to identify 5,400 who will qualify. A sample size of 5,400 was selected for the segmentation because it will provide a robust basis for creating statistically sound segments and because it verges on the upper bound of the AmeriSpeak panel’s feasibility based on discussions with the Director of Business Development at NORC. A larger sample size also provides more responses per segment, which in turn allows for more detailed profiling of subgroups within any given segment. Finally, the larger sample size will provide a more substantial sample of motorcycle riders, the incidence of whom is estimated to be 8% in the general population. Once fielding is complete, NORC will provide the research team with a data file that they will use for analysis and to which they will append the segments that will be created. The research team will create a summary report of findings that will be shared with NHTSA. A cleaned copy of the survey data file will be included as a final deliverable to NHTSA. This data file will not include any personally identifiable information.
B.3 Describe methods to maximize response rates.
In keeping with industry standards, participants will be offered a small incentive to compensate them for their time. The incentive for completing the 60-minute webcam interview will be $100. Participants who qualify and complete the quantitative survey would earn $5 worth of points that could be redeemed for gift cards, a pre-paid debit card or a charitable donation. The $5 value is based on the recommendation from NORC regarding the amount of incentive they typically provide participants for completing a 28-minute survey. The survey will rely on the online panel for the majority of responses. Panelists have previously opted in for receiving and responding to web-based surveys. Those who choose to participate will be incentivized, as noted in Part A (section A9) of this Supporting Statement. They are attuned and receptive to survey invitations. Additionally, as needed, reminders will be sent to participants who have not completed the survey.
Though we estimate that 25.8% of participants who are screened for the research will qualify, we are unable to predict an exact response rate.6 Response rates are subject to a wide range of factors, including methodology (e.g., online versus telephone interviews), topic interest, incentive size and many more. Academic research has estimated survey response rates at around 44.1% on average.7 However, Pew Research has seen response rates as high as 89% when using an opt-in probability panel.8 Response rates may be higher for opt-in panels because they consist of panelists who have volunteered to be given opportunities to take surveys. Our study will use an opt-in probability panel, so we expect most of the sample who is invited to participate and virtually all who qualify to complete the study. However, our study will supplement with other sources and methodologies as needed to interview hard-to-reach groups, which may shift the response rate in ways that cannot be predicted.
B.4 Describe any tests of procedures or methods to be undertaken.
Once programmed, the research team will conduct an internal test of the survey to ensure all questions and responses are programmed correctly. Once internal testing is complete, the first phase of the research will begin. The cognitive test allows for a pilot test of the survey among the target audience and will help discover any potential wording issues where questions (or responses) are unclear or not consistently understood. Additionally, it helps ensure the available response options are adequate.
The goal of the study is to identify and describe at-risk drivers and motorcycle riders ages 21-54. These drivers and motorcycle riders will be screened for relevant demographic (age), attitudinal (attitudes toward drinking and driving/riding), and behavioral (type of license, driving/riding habits, drinking habits) criteria. Drivers and motorcycle riders are not a mutually exclusive group (i.e., we expect most motorcycle riders also to be drivers), so all participants will be screened using the same criteria. We estimate the incidence for motorcycle riders to be 8%9 , which should yield approximately 400 motorcycle riders based on our overall sample n=5,400.
To identify and describe this group of at-risk drivers and motorcycle riders, the research team will utilize advanced multivariate analytics, including but not limited to factor analysis, cluster analysis, including latent class, and other descriptive statistics among the 5,400 cases collected in the quantitative survey.
For the basis of the descriptive data analysis, data tabulation sets will be developed based on a tab plan customized to the final screener and questionnaire. Tabulations will each feature up to 21 banner points. This number of banner points, along with their detailed statistical values, is expected to allow for descriptive comparative analyses of responses from sub-groupings of the sample based on behavioral, attitudinal, and demographic similarities/differences. The individual tabulations will include stubs for all closed-ended data points in the survey, means for Likert and Semantic Differential scale ratings frequency and percentage responses.
Factor analysis will be conducted utilizing SPSS (IBM Statistics) software to examine correlations within sets of variables (across potential segment-defining variables total) reflecting respondents’ self-reported drinking attitudes and behaviors, drinking and driving/motorcycle riding attitudes and behaviors, social behaviors, and personalities. The identification of correlated variables will aid in the selection of variables that are relatively strong in explaining ways in which respondents may cluster. Variables with higher factor loadings from correlated combinations will be considered in cluster modeling.
The cluster modeling will also be conducted utilizing SPSS (IBM Statistics) software. Cluster analyses will be conducted using variables selected after factor analysis and informed by descriptive analysis of responses to related survey questions. These analyses will vary in the number and selection of variables that will define emerging clusters. Several cluster modeling outcomes will be examined to develop four to six (4-6) clusters per model. A cluster model and segmentation approach will be selected, and the related clusters will be represented in an additional set of data tabulations for comparative descriptive analysis of responses to survey questions beyond the final defining metrics.
The final written report will contain detailed findings about each segment. A comparative analysis across segments will be conducted to look for key differences that NHTSA may leverage in strategy moving forward. Descriptive statistics for specific data points and findings will be referenced as appropriate. In the event the detailed data might be useful in the future, the addenda will include data tables of all findings representing all questions in the survey.
B.5 Provide the name and telephone number of individuals consulted on statistical aspects of the design.
The company selected as a contractor for this study is Stratacomm. Stratacomm is a marketing communications and advertising consultancy for NHTSA’s OCCI. For this study, the contractor has selected Heart+Mind Strategies as its research subcontractor. Heart+Mind Strategies is a data-driven research consultancy that specializes in custom research, advanced analytics and communication strategy. This team has extensive experience in both qualitative and quantitative research practices and has consulted with NHTSA on all aspects of the design. The following individuals have reviewed technical and statistical aspects of procedures that will be used to conduct the Alcohol-Impaired Driver/Motorcycle Rider Segmentation Research:
Ben Custer
Senior Solutions Consultant
Heart+Mind Strategies
12110 Sunset Hills Rd, 6th Floor
Reston, VA 20190
703.474.8082
Dr. Carlos Elordi
Chief Data Scientist
Heart+Mind Strategies
12110 Sunset Hills Rd, 6th Floor
Reston, VA 20190
540.588.7548
NHTSA CONTACT
Kil-Jae Hong
Marketing Specialist, Office of Communications and Consumer Information
National Highway Traffic Safety Administration (NHTSA)
1200 New Jersey Avenue, SE
W52-232
Washington, DC 20590
202.493.0524
1 The Abstract must include the following information: (1) whether responding to the collection is mandatory, voluntary, or required to obtain or retain a benefit; (2) a description of the entities who must respond; (3) whether the collection is reporting (indicate if a survey), recordkeeping, and/or disclosure; (4) the frequency of the collection (e.g., bi-annual, annual, monthly, weekly, as needed); (5) a description of the information that would be reported, maintained in records, or disclosed; (6) a description of who would receive the information; (7) if the information collection involves approval by an institutional review board, include a statement to that effect; (8) the purpose of the collection; and (9) if a revision, a description of the revision and the change in burden.
2 2020 Alcohol Impaired Driving (Traffic Safety Facts. Report No. DOT HS 813 294).
3 Ibid.
4 https://markets.businessinsider.com/news/stocks/norc-expands-breakthrough-panel-based-research-platform-welcomes-bruce-barr-to-oversee-amerispeak-omnibus-1027795079
6 This number is derived from the 25.8% incidence of binge alcohol use from SAMHSA, Center for Behavioral Health Statistics and Quality, National Survey on Drug Use and Health, 2018 and 2019. - Table 2.20B – Binge Alcohol Use in Past Month among Persons Aged 12 or Older, by Age Group and Demographic Characteristics: Percentages, 2018 and 2020 See https://www.samhsa.gov/data/sites/default/files/reports/rpt29394/NSDUHDetailedTabs2019/NSDUHDetTabsSect2pe2019.htm#tab2-20b
7 https://www.sciencedirect.com/science/article/pii/S2451958822000409
8 https://www.pewresearch.org/politics/2022/03/24/midterms-2022-methodology/
9 Assumes 8% of households have motorcycles in the U.S., PR Newswire article. See https://www.prnewswire.com/news-releases/us-households-with-a-motorcycle-climbs-to-record-8-percent-in-2018-300783120.html
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Roach, Callie (NHTSA) |
| File Modified | 0000-00-00 |
| File Created | 2023-10-24 |