Part B - OMB Supporting Package-11-15-2012

B. Collections of Information Employing Statistical Methods

The proposed information collection will employ statistical methods (analysis of variance, regression, and correlations) to analyze the data collected from participants. For Study 1, a 5 (CPS Message Positioning Version) X 2 (Time, i.e., pre-post) experiment will use a randomized controlled trial design to examine relative effectiveness of and parent preferences for different electronically randomized to one of five groups (including 4 experimental conditions and 1 control group), and will respond to pre-post surveys. For Study 2, a 4 (CPS Information Combination) X 2 (Time, i.e., pre-post) experiment will use a randomized design to examine the effectiveness of NHTSA’s CPS recommendations in combination with other types of CPS information (i.e., normative and/or installation information). Parent participants will be electronically randomized to one of four experimental groups, and will respond to pre-post surveys. The following sections and the Study Protocol attachment describe the procedures for respondent sampling and data analysis. (See Attachments D through L)

B.1. Describe the potential respondent universe and any sampling or other respondent selection to be used.

This project will conduct two experimental studies. Approximately 540 participants who are parents/caregivers of children ages 0 to 12 years will be involved in the testing across the two sequential studies. Participants will be drawn from two large metropolitan statistical areas on the east coast of the United States. Each study will use a randomized design with repeated measures.

For Study 1, 300 parent participants will be electronically randomized to one of five groups (including four experimental conditions and one control group), and will respond to pre-post surveys. The purpose of the experimental manipulation in Study 1 is to examine relative effectiveness of and parent preferences for different methods of framing child passenger safety (CPS) recommendations. The five experimental conditions for Study 1 are described briefly below and are presented in Appendices E and F. See also the Flow Diagram for Study 1.

1. CPS Message Alternate 1: CPS Recommendations that Focus on Natural Progression: Participants assigned to this condition will view an alternate version of the NHTSA CPS recommendations in which both text and pictures will highlight the natural progression of seat types from birth to teen years. (See Appendix E, Illustration 1).

2. CPS Message Alternate 2: CPS Recommendations that Draw Attention to Premature Graduation. Participants assigned to this condition will view an alternate version of the NHTSA CPS recommendations in which both text and pictures draw attention to premature graduation. Therefore, in addition to specifying recommendations for each stage, this version of recommendations specifically emphasizes the message that counters premature graduation to the next stage. (See Appendix E; Illustration 2)

3. CPS Message Alternate 3: CPS Recommendations that Explain Risk Reduction/ Rationale. In order to be consistent with the risk communication literature and maximize behavior change, this version of the CPS recommendations communicates the risk reduction and rationale (in a lay-friendly, succinct manner avoiding statistics) behind each stage’s recommended restraint configuration, starting first with the basic rationale for occupant restraints, and moving into rationale for specific restraint configurations for the various child sizes. (See Appendix E; Illustration 3)

4. Comparison: Current CPS Recommendations. Participants assigned to this condition will view NHTSA’s current CPS recommendations as depicted in the “Car Seat Recommendations _{for Children” flyer released March 2011 (NHTSA, 2011). (Appendix E; Illustration 4)}

5. Control Group. Participants assigned to this condition will not receive any instructional materials related to CPS during their participation. Rather, these participants will be taken through a computerized pictorial display of various child safety seats and motor vehicles currently on the market and asked to rate their preferences based on style, color, and other characteristics (See Appendix F). To avoid a missed opportunity to educate, control group participants (and all participants) will be given a handout on child passenger safety to take home for reference once they have completed participation in the study (see post-study information handout in Appendix M).

For Study 2, 240 parent participants will be electronically randomized into one of four experimental groups, and will respond to pre-post surveys. The purpose of the experimental manipulation in Study 2 is to examine the effectiveness of NHTSA’s CPS recommendations in combination with two other types of CPS information: normative information and/or installation tips. The four conditions are described below and are included in Appendix G. See also the Flow Diagram for Study 2.

1. Condition 1: CPS Recommendations Alone (No Additional Information). Participants assigned to this condition will view CPS recommendations presented alone, with no additional information about installation or safety norms. (Appendix G; Illustration 1)

2. Condition 2: CPS Recommendations Combined with Installation/Proper Use Tips. Participants assigned to this condition will view CPS recommendations as depicted in Condition 1, presented in combination with succinctly summarized installation/proper use tips for each stage of occupant restraint. (Appendix G; Illustration 2)

3. Condition 3: CPS Recommendations Combined with Normative Information. Participants assigned to this condition will view CPS recommendations as depicted in Condition 1, presented in combination with an additional section that includes normative information regarding compliance with the recommendations. (Appendix G; Illustration 3)

4. Condition 4: CPS Recommendations Combined with Installation Tips and Normative Information. Participants assigned to this condition will view CPS recommendations as depicted in Condition 1, presented in combination with additional sections that include succinct tips for installation/correct use and normative information regarding each stage of occupant restraint and rear seat use. (Appendix G; Illustration 4)

For both studies, recruitment of participants will occur by working with a variety of agency networks (e.g., Safe Kids USA) to disseminate study recruitment announcements through email listserves, social media, childcare and retail store postings, and personal contacts. Through these contacts, we anticipate reaching vast networks of agencies and professionals that directly serve a diverse parent community. Data collection is expected to take place over two 4-month periods in the Fall/Winter of 2012 (Study 1) and Spring/Summer of 2013 (Study 2).

To increase the probability of reaching a diverse sample and economizing staffing resources, participants will be drawn from two large Metropolitan Statistical Areas, Delaware Valley (Philadelphia – Wilmington – Camden) and Hampton Roads (Hampton – Newport News -Norfolk – Virginia Beach - Portsmouth). The Delaware Valley MSA includes 13 counties in 4 States with a population of over 5.9 million people. The Hampton Roads MSA includes 10 cities and 6 counties in two States (VA-NC), with a population of over 1.6 million. These are two socio-economical and culturally diverse areas that range from the rural areas of VA’s Eastern Shore and PA’s farming communities, to large concentrations of people living in urban and suburban settings. Recruitment will occur in locations that will enable reaching potential participants from varied racial/ethnic and socio-demographic groups. Efforts will be made during recruitment of participants to ensure that the sample under study is drawn from the larger population, and that the two studies include a variety of income levels, education levels, and racial/ethnic groups (at least Black, White, Latino) across the study conditions.

Interested participants will call the phone number on the recruitment materials (an email address will also be an option), at which point a member of the research team will briefly describe what is involved in participation, will ask some demographic questions, and will ask whether or not the participant is comfortable reading English-language text displayed on a computer screen (see phone script in Appendix K). For qualifying participants, a study appointment will be scheduled. Preliminary demographic questions (e.g., age, age of children, etc.) will be asked to ensure the participant is appropriate for the study. Enrolled participants will receive $50 compensation (e.g., retail store gift card) following participation in this study, which is expected to take approximately 60-75 minutes.

Power analysis for Study 1 indicated that a sample of 196 participants is needed to detect small effect sizes above Cohen’s d = 0.25 with a statistical power of 0.80 at the standard significance level of α = .05. Therefore, the sample size of 300 participants will be ample to detect group differences in Study 1. Power analysis for Study 2 indicated that a sample of 179 participants is needed to detect small effect sizes above Cohen’s d = 0.25 with a statistical power of 0.80 at the standard significance level of α = .05. Therefore, the sample size of 240 participants will be ample to detect group differences in Study 2. Equal variance between groups is expected due to random assignment. Effect sizes for similar studies of intervention messages with smaller sample sizes and fewer experimental groups have been in the medium to large range (Cohen’s d = .50 to 1.68) (Will, Sabo, & Porter, 2009).

B.2. Describe the procedures for the collection of information.

This research utilizes a secure web-based study protocol in which participants will come to a computer lab and view a series of user-friendly computer screens that will automatically lead them through the informed consent document, pretest measures, study materials specific to condition assignment, and post-test measures at the participant’s pace. Prior to the program being launched for use with participants, the web-based study apparatus will be subjected to a number of test-runs by all study personnel (including the NHTSA contract manager), as well as several of their staff members and research colleagues. During these preliminary checks, individuals will undergo identical procedures as actual participants, allowing for thorough manipulation checks of all question/answer sequences and automated data storage procedures. Any typos, errors, or software glitches will be noted and corrected prior to enrolling participants. These preliminary checks will ensure the entire study apparatus and the sequence of screens is working properly prior to the start of data collection.

Enrolled participants will be asked to arrive at a designated computer lab/center (university lab or commercial learning center) at their appointment time to participate in the study. This study will use a secure web-based study protocol in which participants will view a series of user-friendly screens that will automatically lead them through the informed consent document (detailing the logistics of the study, the study’s duration, their rights as a participant, and remuneration for their participation), pretest measures, study materials specific to condition assignment, and post-test measures at the participant’s pace. As participants respond to survey questions, they will simply check off their desired response(s) in the same manner as they would on paper. (The survey questionnaire is included in Appendix H.)

A study assistant will be present at the computer center to assign participants to a computer station, help participants log into the study using a secure code, orient them to the process of completing study materials, and answer any questions as they arise. Participants will complete study materials individually, but we are expecting to be able to run as many as 25 subjects simultaneously at different computers in the computer lab/center.

At the conclusion of study participation (which is expected to take approximately 1.25 hours), participants will receive information on child passenger safety (Appendix M) and receive their compensation ($50 retail gift card). The team has compensated participants in the same manner on other research projects and the amount and nature of the compensation has been found to be appropriate for participants and effective in recruiting participants; there have been no reported negative issues with this arrangement.

Participant responses will be automatically entered into a web-based data collection system. The data will initially be checked for missing data, outliers, ensuring normality and linearity of all dependent variables.

B.2.1 Child Passenger Safety Survey Items

As the participants click through the screens in the study software, several measurement scales will be used to assess appropriateness of restraint selection, knowledge of restraints, perceived efficacy and threat, attitudes and intentions, judgments of relevance and acceptability of the CPS information, and sample demographics. To accurately assess changes in knowledge and perceptions after exposure to the independent variable, most measures will be asked both at pretest and posttest. The exceptions are the demographics questions and judgments of relevance and acceptability, which will be asked only once at posttest (see survey in Attachment H). Justification and origination of each survey subscale are provided below.

Restraint Selection Task: Questions 1-8 (pre and post)

A key dependent variable for Study 1 is proper child restraint selection; thus, a Restraint Selection Task was developed that provides participants with a series of specific scenarios that ask the participants to select an appropriate restraint, direction to face, and/or vehicle row given specific age/weight/height information for a hypothetical child. This 8-item knowledge measure uses a multiple choice response format (providing an item score of correct/incorrect and a total number correct score for each participant). This measure was adapted from a similar existing field-tested measure (Snowden et al., 2008).

Knowledge of Restraint: Questions 9-23 (pre and post)

To gauge immediate changes in knowledge of child passenger safety, as well as differences in knowledge among the groups, a 15-item assessment of parental knowledge will be conducted at both pretest and posttest. This measure uses a Likert-type response format and was tailored for this study from existing field-tested and validated measures used in past research by Snowden and colleagues (2008, 2009), and Will and colleagues (2009). Many questions originated from the Co-Investigators Kids in Kars Survey which has excellent internal consistency (α = .92) and measures parents’ knowledge of and attitudes toward recommended guidelines for restraint use (Will, Sabo, & Porter, 2009).

Perceptions of Efficacy and Risk: Questions 24-39 (pre and post)

The Risk Behavior Diagnosis Scale (RBDS) (Witte, Cameron, McKeon, & Berkowitz, 1996) will be used to assess perceived efficacy and risk. The RBDS is a 12-item template survey designed to be tailored for evaluation of any health or safety message. The efficacy subscale (internal consistency α = .79, 6 items) assesses participants' perceptions of response efficacy (i.e., confidence that the recommended actions/restraints will work to prevent injuries) and self-efficacy (i.e., confidence in one’s ability to follow child passenger safety recommendations). The threat subscale (internal consistency α = .82, 6 items) assesses participants’ perceived risk by measuring susceptibility to and severity of negative consequences from inappropriate child occupant protection. Four additional “severity” items were added to properly asses perceptions of severity of the threat at each of the 4 stages of occupant protection. This, the final RBDS contains 16 items, all of which use a 5-point Likert-type response scale (1 = strongly disagree to 5 = strongly agree).

Attitudes and Intentions: Questions 40-56 (pre and post)

Participant’s more general attitudes and intentions regarding child passenger safety will be assessed via an 8-item attitudes subscale adapted from the previously mentioned Kids in Kars survey (Will, Sabo, & Porter, 2009), and a 9-item stated intentions subscale. Stated intentions and attitudes will be assessed to gauge participants’ disposition regarding what is recommended for child occupant protection irrespective of their knowledge. Both subscales use a Likert-type response format (1 = strongly disagree to 5 = strongly agree).

Installation: Questions 57-64 (Study 2 only; pre and post)

Parents will be asked about installations and use of safety restraints via an 8-item installation subscale. Parents will be asked questions directly relating to the correct installation procedures of infant and toddler safety seats, safe use of boosters and seat belts, and the correct use of harness straps. The questions will be used to gauge the parent’s knowledge of the correct installation and use of safety restraints. The subscale uses a Likert-type response format (1 = strongly disagree to 5 = strongly agree).

Judgments of Relevance and Acceptability: Questions 65-74 (post test only)

At posttest, parents in each condition will be asked their opinions about the CPS materials. A 10-item questionnaire was developed that uses a 4-point Likert-type response format to assess participants’ judgments of quality and acceptability of the CPS information presented. Specifically, they will be asked to rate the CPS information on a variety of factors, including but not limited to style, amount of information, clarity, and likelihood for motivating behavior change.

Demographics and Other Participant Information: Questions 75-86 (post test only)

Demographic information will also be collected at the posttest, including but not limited to participant’s age, sex, race, ethnicity, education level, income level, and number of and ages/sizes of children. Information specific to child passenger safety will also be assessed, including types of child restraints being used currently (by child), sources of information about safely transporting children, whether or not they have had their child restraints inspected by a CPS technician, and their preferred communication channels (e.g., print, television, radio, electronic) for receiving child passenger safety information.

B2.2 Data Analysis

Data analysis for Study 1 will determine which CPS recommendation version produces the maximum desired outcome (e.g., correct CPS restraint selection, increased knowledge of restraints, enhanced perceptions of efficacy and risk). Data analysis will begin with data cleaning such as checking for missing data and outliers and ensuring normality and linearity of all dependent variables. Next, bivariate correlations will be conducted on all dependent variables to assess their relation to each other and to determine the type of inferential statistics to conduct. Specifically, if dependent variables are moderately inter-correlated, then a 2 (Time) x 5 (CPS Recommendation Version) MANOVA with follow-ups, post-hoc test for group, and simple effects analyses (if the interaction is significant) will be conducted. If dependent variables exhibit inter-correlations near zero or are highly inter-correlated, then multiple 2 (Time) x 5 (CPS Recommendation Version) ANOVAs with post-hoc test for group and simple effects analyses (if the interaction is significant) will be conducted. In the event that the data are found to violate the assumptions of normality that are necessary for parametric tests, data transformations to improve normality or appropriate nonparametric tests, such as the Friedman test and follow-up Multiple Comparison Tests, will be performed instead. Additionally, the study team will conduct several secondary analyses, including a multiple regression for CPS restraint selection to determine the CPS message that best predicts correct restraint selection. Also, variables such as perceived relevance and accuracy of CPS information, age, sex, race, education, income, number of children, and ages of children may be used as covariates to eliminate potential confounding variables and better explain variance in the analyses.

Analyses for Study 2 will determine which CPS information combination produces the maximum desired outcome (e.g., correct CPS restraint selection, increased knowledge of restraints, increased perceptions of efficacy and risk, increased knowledge of common mistakes). This will initially consist of data cleaning such as checking for missing data and outliers and ensuring normality and linearity of all dependent variables. Next, bivariate correlations will be conducted on all dependent variables to assess their relation to each other and to determine the type of inferential statistics to conduct. Specifically, if dependent variables are moderately correlated, then a 2 (Time) x 4 (CPS Information Combination) MANOVA with follow-ups, post-hoc test for group, and simple effects analyses if the interaction is significant will be conducted. If dependent variables exhibit correlations near zero or are highly correlated, then multiple 2 (Time) x 4 (CPS Information Combination) ANOVAs with post-hoc test for group and simple effects analyses if the interaction is significant will be conducted. In the event that the data are found to violate the assumptions of normality that are necessary for parametric tests, data transformations to improve normality or appropriate nonparametric tests, such as the Friedman test and follow-up Multiple Comparison Tests, will be performed instead. Additionally, the study team may conduct several secondary analyses, including a multiple regression for CPS restraint selection to determine which CPS information combination best predicts correct restraint selection. Also, variables such as perceived relevance and accuracy of CPS information, age, sex, race, education, income, number of children, and ages of children may be used as covariates to eliminate potential confounding variables and better explain variance in the analyses.

B.3. Describe methods to maximize response rates and to deal with issues of non-response.

This is a randomized control experimental design. Participants have volunteered to be in the study. If they are eligible, then they will be randomly assigned of the conditions. Non-response is not anticipated to be an issue, as the participants have already agreed to be in the study.

As participants respond to survey questions (Appendix H), they will simply check off their desired response(s) in the same manner as they would on paper. Manipulation checks (e.g., page and survey view times) will be electronically embedded to ensure that participants attend to the messages and read questions versus randomly choosing answer choices.

The data collected from all participants will be anonymous. No person’s name or other personal identifier will be stored with the data (a coding process will be used to link pre-post data that does not identify the participant). The software package used to collect the web-based survey data will automatically send the survey responses to a securely housed and password-protected statistical database (with individual cases for participants and separate variables for each data element) for later analysis with SPSS statistical software.

B.4. Describe any tests of procedures or methods to be undertaken.

Most of the questions proposed on the Child Passenger Safety Questionnaire have been used in previous research and do not need to be pre-tested. The Installation Items (Questions 57-64; Study 2 only) are factual questions about the information that was presented. Questions 65-74 (post-test only) were developed to assess the quality and acceptability of the information presented. These items have been screened and discussed for content and face validity with experts in the fields of scale development and child passenger safety.

B.5. Provide the name and telephone number of individuals consulted on statistical aspects of the design

The following individuals have reviewed technical and statistical aspects of the study design:

Kelli England Will, Ph.D.

Associate Professor & Clinical Psychologist

Division of Community Health and Research

Williams Hall, Department of Pediatrics

Eastern Virginia Medical School

855 West Brambleton Avenue

Norfolk, Virginia 23510-1001

Phone: 757-668-6449

E-Mail: willke@evms.edu

Amy M. Perkins, M.S.

Biostatistician, Department of Pediatrics

Eastern Virginia Medical School &

Children’s Hospital of the King’s Daughters

601 Children’s Lane

Norfolk, VA 23507

Phone: 757-668-8221

E-Mail: amy.perkins@chkd.org

Maria Vegega, Ph.D.

Chief, Occupant Protection Division

National Highway Traffic Safety Administration, NTI-112

1200 New Jersey Ave., S.E., West Bldg. 44-301

Washington, D.C. 20590

Phone: 202-366-2668

Email: maria.vegega@dot.gov

6