SSB_rev_2015-3-23-15

HHS/CDC/NCIPC

SUPPORTING STATEMENT FOR

OMB INFORMATION COLLECTION REQUEST

Part B

March 23, 2014

Improving the Understanding of Traumatic Brain Injury through
Policy and Program Evaluation Research

Supported by:

Department of Health and Human Services

Centers for Disease Control and Prevention

National Center for Injury Prevention and Control

Division of Unintentional Injury Prevention

Government Project Officers:

Point of Contact for OMB:

Lisa Garbarino

4770 Buford Hwy NE

Atlanta, GA 30341, Mailstop F-62

Telephone: (770) 488-1496

Fax: (770) 488-3551

Electronic Mail: LGT1@cdc.gov

Table of Contents

B. Collection of Information Employing statistical procedures

1. Respondent Universe and Sampling Methods

2. Procedures for Collection of Information

3. Methods to Maximize Response Rates and Deal with Nonresponse

4. Test of Procedures or Methods

5. Individuals Consulted on Statistical Aspects and on Data Collection and Analysis

References

B. Collection of Information Employing Statistical Methods

B.1. Respondent Universe and Sampling Methods

The proposed study will examine boys and girls ages 14–18 who play soccer in competitive leagues organized by the US Youth Soccer Association (USYSA). The study will review State legislation and the variation in legal requirements across State laws as well as USYSA association policies as they apply to their associations in different States to form two groups for comparison, described below.

We want to test the hypothesis: that young athletes playing on teams governed by a policy that requires RTP only after a written release from a medical professional trained in concussion evaluation and management (Robust RTP) are less likely to RTP with symptoms than comparable athletes playing on teams where there are no specific policies governing RTP (No RTP).

USYSA serves more than 3 million boys and girls ages 5 to 19 and is made up of 55 member State associations (1 in each State and 2 each in California, New York, Ohio, Pennsylvania, and Texas) that are divided into 4 regions. It sponsors both recreational and highly competitive soccer leagues that are outside of public or private school athletic associations. Roughly 30 percent of USYSA athletes are ages 14 or older. From a sample of soccer teams participating in USYSA competitive leagues during the fall of 2015 (see Table 1), this study will select a sample of soccer coaches and a dyad sample of athletes ages 14–18 and their parents using a stratified, multistage random sampling approach.

Preliminary analyses found that some USYSA State associations were following concussion evaluation and management policies that were different from the prevailing State sports-related concussion legislation. This occurred because many of the State laws only apply to school-based sports, leaving non-school based youth sports organizations such as USYSA free to implement their own policies if they chose to do so. Consequently, for our study, the soccer teams have been classified into one of two comparison groups based on RTP requirements as mandated by either the State law or the State association policy that governs their league’s play. In States where non-school youth sports programs are not subject to their State’s law requirements, teams were included in the strata most consistent with the policies adopted by the State USYSA soccer association. The two comparison groups include:

1. Teams playing in States governed by State legislation or a State association policy where there is no specific requirement for youth athletes returning to play after a concussive injury event. This group would serve as the control (Stratum 1–No Specific RTP; 14 State associations).

2. Teams playing in States governed by State legislation or a State association policy that require approval by a healthcare professional trained in concussion evaluation and management before an athlete is allowed to return to play after a concussive injury event. This represents the most robust comparison group (Stratum 2– Specific RTP; 13 State associations).

Table 1. State Association Sample

State Association Sample
Stratum 1 (No Specific RTP)	Stratum 2 (Specific RTP)
Alaska* Arkansas* California, South* Delaware* Georgia* Iowa* Kansas* Missouri* New York, East* North Carolina* Oklahoma* Texas, North* Texas, South* Wyoming†		Arizona† California, North* Colorado† Florida† Illinois* Louisiana† Maryland† Michigan† Nebraska† South Carolina* Utah† Washington* Wisconsin*

*USYSA State Association governed by association policy

†USYSA State Association governed by State law

Figure 1 below illustrates the proposed sampling strategy. The study results will be generalizable only to the population of boys and girls ages14–18 participating in competitive USYSA league play and their coaches and parents. The results will not be nationally representative of all soccer athletes ages 14–18 and their coaches and parents. State associations that do not sponsor regular league play during the fall season were removed from the sampling frame. A list of the State associations included in the study is provided in Table 1.

Figure 1. TBI Study Multistage, Stratified Sampling Approach

The goal of the sampling strategy is to produce two samples:

1. A sample of 180 coaches of non-school soccer association teams for boys and girls ages 14–18.

2. A sample of 1,518 athlete-parent dyads, each consisting of one boy or girl ages14–18 and his or her parent or guardian. There will be 759 male athlete-parent dyads and 759 female athlete-parent dyads.

Using the multistage, stratified sampling approach, teams will be randomly selected. Soccer clubs will be stratified into one of two groups according to the return-to-play requirements of the State law or State association policy (see above for description). All soccer clubs within a State following the same RTP requirements will be placed in the same stratum. Systematic random sampling will be used to select teams within each stratum. For maximum precision when comparing outcomes across the strata, each stratum will have roughly the same number of teams sampled.

Recruitment Process

Before data collection begins, we will conduct several activities to promote and raise awareness for the study. We will begin by engaging USYSA leadership at the national, regional, and State levels to garner their support. As part of our formal partnership with USYSA, we already have an endorsement letter from the national organization. This letter (Attachment I) will be part of the outreach and recruitment materials we will distribute to the soccer clubs to encourage participation. The research team will also promote awareness of the study in the coming months in a variety of ways. We will work with USYSA officials and the Centers for Disease Control and Prevention (CDC) to include a study announcement on their Web sites, Facebook pages, and Twitter accounts before the start of recruitment. We will also include a description of the study in USYSA newsletters or magazines sent to members.

Once the soccer teams have been selected for the study, we will obtain information about them from USYSA, online from the club’s Web site, or by contacting the club leadership.

We will send a short personalized e-mail to the club president and to the coach and parent liaison of each selected team letting them know that their team has been selected and that they can expect to receive information about the study (see draft example in Attachment P). We will then send a packet that will include the endorsement letter from USYSA, a description of the study, and informational flyers that the parent liaison may distribute to the team parents (Attachments I and J). People receiving these packets will also be invited to attend a Webinar we will host, where they can view a brief presentation and voice any questions or concerns they may have about participation.

Information about the Web-based preseason survey and the weekly interactive voice response (IVR) surveys will be distributed to the coach and each parent on selected teams (see description of surveys in section A.1 and survey samples in Attachments C through H). This will include the Web address to access the survey and a unique identification code that will be used to register their contact information. Each parent will be asked to give his or her name and contact information to be used for follow up and weekly IVR reminders. This will allow the study team to link the ID number to a person, but the information will only be used for follow-ups and reminders. The data from the coach and parent surveys will be kept in a separate database apart from any identifying information and will only be identifiable by each participant’s ID code. Following parent consent and disclosure of athlete contact information, athletes will follow a similar procedure to participate.

Coaches. Our target enrollment rate for coaches in the preseason, online baseline survey is 75%--i.e. 270 coaches out of an eligible pool of 360. For this study, coaches will only be asked to complete the preseason survey (see sample in Attachment C). In the Lystedt pilot study, researchers were able to enroll 270 out of 496 coaches as participants in the preseason online, baseline survey (a 54% participation rate). We anticipate our study enrollment (participation) rates for the current study will be higher than the pilot study because of two factors not available to the Lystedt pilot study researchers:

1. The sponsorship of the research by the Centers for Disease Control and Prevention (CDC); and,

2. The USYSA—the governing association for the soccer teams led by these coaches—has agreed to be a full and active partner with CSR in the outreach, education and recruitment of study participants. The research team has entered into a formal partnership with the USYSA leadership (Attachment M). The national organization, along with its board of directors, has agreed to allow us access to its membership and will provide support and guidance throughout the course of the study. In addition to support at the national level, we will engage the regional and State-level leadership to garner their support and cooperation among the clubs and coaches. We will include a letter of support from the organization when we initially contact coaches. We believe that the organizational support will ease any concerns coaches may have about participating in the study.

Literature on survey response rates has shown that social networks, sponsorships, and subject salience are important influencers of survey response rates. Surveys sponsored by governments, trade associations or affinity groups tend to have higher response rates compared to surveys sponsored by researchers outside of the subpopulations social networks. Surveys addressing subjects of particular interest or salience to the target study population also have higher responses rates than those that lack such salience. (See, Suan Bartholomew and Anne D. Smith, Improving Survey Response Rates from Chief Executive Officers in Small Firms: The Importance of Social Networks, Entrepreneurship Theory and Practice, Vol. 30, Issue 1, 83-96, January 2006; L. Kanuk and C. Berenson, Mail Surveys and Response Rates: A literature Review, Journal of Marketing Research, Vol. 12, No. 4(Nov. 1975), 440-453; Eleanor Singer, Nonresponse Bias in Household surveys, Public Opinion Quarterly, Vol. 70, No. 5, 2006; Frederik Anseel, et al,, Response Rates in Organizational Science, 1995-2008: A meta-analytic review and guidelines for survey researchers, Journal of Business Psychology, September 2010, Volume 25, Issue 3, 335-349; KB Sheehan, Response rate variations in e-mail surveys: an exploration; Journal of Advertising Research (1999))

While we believe that these factors will greatly enhance the survey participation rates, the researchers chose to be more conservative in their IRB submission. Thus the proposed participation rates discussed in the IRB were intentionally estimated to be low in order to ascertain the size of effects that could be discerned if study participation was substantially lower than expected. Subsequently, however, USYSA has indicated their increased interest in being an active partner in the outreach and recruitment of survey participants—this was not as clear when we submitted the original IRB request for review and approval. This gives us greater confidence that our higher participation and enrollment estimates are achievable.

Athlete-parent dyads. Given the sampling strategy, we will select 360 teams. The study will collect a baseline survey and 10 weekly reports from the sample of athlete-parent dyads over the course of the fall 2015 playing season (see samples of surveys in Attachments D and F). All athletes and their parents on selected teams will be invited to complete the online preseason baseline survey and, if they complete this survey, to participate in the 10 weekly surveillance reports during the 2015 fall soccer season. Eligibility for the weekly surveillance portion of the study will be contingent upon completion of the preseason survey. This is the same approach taken in the Washington State pilot study on the Lystedt Law, where study participants were required to complete the preseason baseline survey to participate in the weekly surveillance surveys. Researchers conducting the Lystedt Law evaluation enrolled 82 percent of soccer athletes (290 out of 352 eligible) in their study. However, those researchers were able to attend team meetings to explain the study and enroll athletes and parents in person.

Because the average soccer team includes 15 athletes, we expect an eligible sample of 5,400 athlete-parent dyads—2,700 per stratum. While we anticipate that our inability to meet face to face with potential survey participants will depress participation rates from the 82% found in the Lystedt pilot study we believe that the strong partnership and sponsorship of this study by both CDC and the USYSA, and the salience of this issue to athletes and parents will contribute to increasing study enrollment and participation. Given these two competing forces, we anticipate that our response rates for the athlete-parent dyads for this study will be 75% or higher. (See citations above in coaches section) In addition, the research team intends to increase its investment in outreach and recruitment, Participation in the study will be triggered by formal written consent by a parent or guardian and written assent by athletes under the age of 18 and formal written consent from those athletes aged18.

For the preseason online survey, participation and response rates are the same. However, the research team expects to lose some athlete-parent participation on the weekly surveys due to attrition (e.g., athletes forget or are unable to continue playing with their team) and diminishing interest. If no response is received for 5 consecutive weeks, the respondent will be dropped from the study. Furthermore, if an athlete ceases to play with the selected soccer team for any reason, the study team will not follow up with that athlete. Taking into account attrition at each stage of the study, we expect the participation rate for the weekly surveillance surveys to be 60 percent of the 5,400 eligible athlete-parent dyads.

Based on the experience from the Lystedt Law study, we expect response rates on the weekly surveillance surveys to be 75 percent or higher. The researchers in the pilot study found that initial response rates for the weekly surveys ranged from 33 percent to 54 percent. By making follow-up calls, the research team was able to improve the weekly response rates to between 49 percent and 72 percent. Again because of the salience of the issue, and the partnership with USYSA we believe that weekly response rates can be kept high.

We anticipate that our natural participation rate will be lower because we cannot personally recruit participants. However, we will compensate for this by gaining the support and assistance of the USYSA leadership and study notification via existing social media channels. In addition, we will employ additional communications channels the pilot study did not use extensively. Specifically, we will implement the following outreach, recruitment and follow-up strategy to educate athletes and their parents about the study, boost the participation and response rates, and reduce attrition:

• Recruitment efforts:

  • We will use the existing USYSA team communication infrastructure to recruit study participants.

  • We will incorporate a robust social media strategy such as posting reminders to USYSA and individual team Web sites, USYSA and individual team Facebook pages, and any other relevant site or method teams use to distribute information about the study.

• Follow-up strategies

  • We will call enrolled nonresponders who do not submit the weekly reports.

• We will send individual e-mail reminders to enrolled study participants.

• We will send general study reminders that do not include information about study participants, through the existing USYSA communication infrastructure.

We strongly believe these additional efforts will greatly improve our response rates.

Study Population and Samples

COACH SAMPLE
Total Number of sampled Teams	360
Number of Coaches Selected per Team	1
Initial Sample	360
Preseason Sample Assuming 75% Participation Rate	270
ATHLETE-PARENT SAMPLE
Number of Teams	360
Number of Athletes per Team	15
Total Eligible Population of Athlete-Parent Dyads Invited to Participate	5,400
Preseason Athlete-Parent Sample Based on 75% Participation Rate	4,050
Weekly Surveillance Sample Assuming 60% Participation Rate	3,240

B.2. Procedures for Collection of Information

In the recent Lystedt Law study, Rivara et al. found the concussion rate during girls soccer games to be 10.0 cases per 1,000 athlete-games and the concussion rate during practices to be 0.8 cases per 1,000 athlete-practices. Therefore, the overall concussion rate for girls games and practices was 5.4 concussions per 1,000 athlete-exposures [1]. Moreover, the UW research team found that the cumulative concussion incidence rate for athletes was 11.1 percent per season. The Lystedt Law study also found that 69 percent of concussed athletes continued to play with symptoms.

Based on the size of our expected sample, we expect to collect data on 30,360 athlete exposures and 1,518 athletes. Because the State of Washington’s RTP policy places them in stratum 2, this group will be used as the reference group. Using the UW study as a benchmark, we anticipate the total number of concussions across the entire sample to be 5.4 per 1,000 athletic exposures, and the cumulative incidence of athletes with concussions to be 11.1 percent [1]. The number of athletes who might be expected to play while still symptomatic is 84. Table 2 presents our hypotheses.

Table 2. Sample and Effect Sizes by Stratum for Youth Sports TBI Study
	Stratum 1 = No RTP Requirements	Stratum 2 = RTP with medical approval from TBI trained practitioner	Total
Sample Size	759	759	1,518
Athletic Exposures	15,180	15,180	30,360
Rate/1,000 Athletic Exposures	5.4	5.4
Number of Concussions	54	54
Cumulative Incidence Rate (%)	11.1	11.1
Cumulative Incidence Number	84	84
RTP with Symptoms	Null: X = 58 cases Alt: X > 58 cases	58

Although the data collection will provide information on a variety of variables related to concussions, the primary outcome indicator of interest is in determining whether the relative risk of an athlete RTP with symptoms after a concussion injury varies between the two groups of States.

The most important consideration in determining sample sizes for this sort of study is whether our proposed samples will be sufficient to detect differences between the two groups in the proposed outcome indicator. To address this issue, the research team conducted two power analyses:

1. The first assumed that there are no nesting or clustering effects because athletes would be selected from the same teams.

2. The second adjusted the samples using the actual Intraclass Correlation Coefficient (ICC) calculated from the Lystedt Law study.

An analysis of the Lystedt Law study found that the Intraclass Correlation Coefficient (ICC) was .111. This was used to determine how large an effect size would be detectable using the proposed sample, assuming that there would be no nesting or clustering effects in the data.

Power Analysis Assuming no Nesting or Cluster Effects¹

For the power analysis we chose the more conservative estimates of sample size. However the figures show the the studies ability to detect effect sizes rise with increases in sample size.

Coaches Sample

The proposed samples for the study are based on selecting participants from an eligible pool of athletes, parents, and coaches from 360 teams (180 boys’ teams and 180 girls’ teams). The head coach of each selected team will be invited to participate in the onetime online preseason baseline survey. We anticipate based on our sampling assumptions that 50 percent of the invited team coaches will choose to participate in this survey—90 in each strata. The response rate for most national online surveys is around 25 percent. However, the literature indicates that response rates can be improved with increased follow-up and when working with a highly motivated population. In fact, with highly motivated populations, response rates can easily reach 85 percent. Because we are partnering with USYSA, we expect that coaches, athletes, and parents will be highly motivated to participate in a study endorsed by the association. In addition, the research team intends to invest additional resources in follow-up to improve the response rates.

Because concussion knowledge is critical to a coach’s ability to evaluate whether an athlete has sustained a concussion and to respond appropriately, the absolute percent difference in concussion knowledge, especially understanding of RTP, was used to measure effect size differences between the two groups. Based on the Lystedt Law study, 95 percent of coaches had a perfect score on the concussion knowledge tests. We assume that coaches in the Robust RTP requirements group will also have the highest scores on the concussion knowledge test compared with coaches in the No RTP requirement group. Figure 2 below presents the power analysis for the coaches’ sample using the absolute percent difference between groups of coaches in concussion knowledge as the effect size.

Figure 2. Power Analysis for Coaches Sample

Given the high scores on the coaches’ concussion knowledge test in the Washington State pilot study, the sample must be large enough to test the following hypothesis:

• Null Hypothesis: Group 1 Concussion knowledge (reference group) = Group 2 Concussion knowledge

• Alternative Hypothesis: Group 2 Concussion knowledge is > Group 1 Concussion knowledge.

The results of the analysis indicate that even using an assumption of 90 percent power, the proposed coaches sample should be able to detect effect size differences of 16 percent or more. Indeed, an extended analysis indicates that the proposed coaches sample is large enough to detect similar differences even if two-sided hypotheses are tested.

Athlete-Parent Dyad Sample

Soccer teams have an average of 15 athletes. All athletes and their parents on a selected team will be invited to participate. Because we are collecting data on dyads and not teams, no minimum team participation rate will be required. Based on our assumption about the number of teams that will provide eligible player-parent dyads, the eligible sample will be 5,400 athlete-parent dyads. Our proposed sample is 1,518 athlete-parent dyads; 759 per strata; 50 percent boys and 50 percent girls. This means that the target study participation rate is 28 percent. Average response rates for national online surveys are around 25 percent. We believe that the increased resources that will be devoted to outreach, recruitment, and follow-up along with the endorsement of USYSA will help to improve our participation rates above national averages. Based on the Lystedt Law study, the anticipated number of cumulative concussion cases for this sample will be 168 (i.e., roughly 11.1 percent); and the percentage that are expected to RTP with symptoms in the reference group (the group with the most robust RTP requirements) is 69 percent.

Athletes and parents will be eligible to participate in the study regardless of their coach’s participation status. However, we will only include athlete-parent dyads in the study. Therefore, one parent or guardian must consent to participate in the sample for each selected athlete. Figure 3 below provides an analysis of the sample size for concussion cases and the effect sizes detectable using three different assumptions about power–.70, .80, and .90. The key assumption is that there are no nesting or cluster effects around teams. The graph shows that the proposed sample should allow the detection of effects based on the relative risk of RTP with symptoms of 1.23 or higher.

Figure 3. Results of Unadjusted Power Analysis

However, given the calculated ICC associated with team membership of .111 in the Washington State pilot, the sample sizes may need to be adjusted upward to achieve the potential effect sizes presented in Figure 3 above. The ICC of .111 indicates that there are modest nesting or clustering effects within teams that were present in the Washington State pilot study. Figure 4 below shows that the proposed sample would be expected to lose some precision. However, the loss in precision appears to be modest. Accounting for the ICC of .111, the proposed sample of concussion cases of 169 should allow for the detection of effect sizes around 1.27 and larger.

Figure 4. Power Analysis adjusted for ICC = .111

Our data collection strategy will consist of a number of data collection methods that are described in this section. We will obtain data on concussion knowledge, behaviors, and attitudes; concussive events, including the presence and severity of symptoms; use of Heads Up or other concussion education or training programs and materials; and management of RFP and RTP. These will be voluntary self-report data from two national subsamples: club soccer coaches and dyads consisting of boys and girls ages 14–18 who are club soccer athletes and their parents or guardians. The two subsamples will be drawn from coaches and athlete-parent dyads participating on USYSA member teams before the start of the fall 2015 playing season. The coach, athletes, and parents or guardians of athletes will be approached to complete the preseason survey following Institutional Review Board-approved consenting/assenting guidelines.

Prevalence of RTP with symptoms will be examined using athletic exposure data gathered through weekly surveillance reports and through data gathered in follow-up interviews with injured athletes and their parents. This outcome will be expressed as a percentage of injured athletes. Variation in the level of RTP with symptoms is expected to vary across strata. Given the power analysis conducted based on calculations drawn from the Washington State pilot study, we estimate that our samples will be able to detect effect sizes of roughly 23 percent –– 27 percent or larger with Power: 1 – β = 0.80 even assuming the modest nesting effects implied by an .111 ICC calculated in the Lystedt Law study (see Figures 3 and 4 above).

Data will be collected from all participants (coaches and athlete-parent dyads) in the study at baseline prior to the start of the soccer season and weekly for 10 weeks from the athlete-parent dyads.

Coaches. Coaches of selected teams will receive an invitation describing the onetime online preseason survey (Attachment K). When logging on to complete the survey, coaches will be presented with an informed consent document (Attachment N) and asked to check a box to indicate their consent to participate.

Athlete-Parent Dyads. Because most of the athletes selected will be minors, parental consent will be required for their participation. Parents and athletes will be informed about the survey through at least one of the following ways. First, the researchers will work with the coach and/or team parent of a selected team to post an announcement on the team Web site and Facebook page, if one exists. Second, a letter about the survey will be e-mailed and/or distributed by the parent liaison to all parents of athletes on selected teams (Attachment K). Parents will be given the name and e-mail address of a contact person on the research team to request a copy of the survey or to ask questions about survey participation.

The athlete and parent preseason surveys will be administered online, with the option to complete a hard copy of the questionnaire. Information about the Web-based preseason survey and the weekly IVR reports will be distributed to each parent on selected teams (see survey samples in Attachments E and G). Each parent will be asked to provide his or her name and contact information to be used for follow-up and weekly IVR reminders by following a link included in the survey invitation. Parents will also be asked to provide contact information for their athlete and to provide consent if their athlete is under age 18. Using the parent provided information, athletes will be contacted directly to obtain their consent/assent.

Each athlete and parent will have a randomly generated ID number that he or she will use to complete the preseason and weekly surveillance reports. This will allow the study team to link the ID number to an individual but will only be used for follow-ups and reminders. Their survey responses will be kept in a database and will only be identified by their ID code to protect the respondent’s privacy and confidentiality. The file linking the respondent’s identity to the ID number will only be accessible to the contractors for data collection and follow-up purposes.

The weekly surveillance reports will be collected using an IVR system. The IVR system will automatically notify the study team by e-mail of positive responses about injuries that could indicate concussion symptoms. Research assistants will then call the athlete and parent separately within 24 hours to determine the date of the injury; whether it was during a practice or a game; the severity rating of symptoms using a standardized, validated assessment of concussion based on a 0 to 6 scale (higher scores indicating more severe symptoms) on the day of injury and in the 24 hours before interview; whether the athlete continued to play with symptoms; and if and to whom the athlete reported symptoms (see interview protocol in Attachment H). We will also ask about medical care for the injury and instructions on return to play. For athletes who report continued concussion symptoms at the time of the initial call, both athletes and parents will continue to be called weekly until symptoms abate. Experience from the Lystedt Law study indicates that on average, only one follow-up interview was necessary.

All injury interviews and follow-up assessments will be reviewed by one of the physician members of the research team. The research team physician will review the participant’s responses, and the determination of a potential concussion will only be assigned if:

• the athlete experienced more than just headache symptoms,

• if symptoms were rated as being more than mild, and

• if the symptoms lasted more than 1 day.

If a potential concussion determination is made, the research team will notify the parent and athlete that the signs and symptoms reported meet the criteria for a potential concussion and suggest that any medical concerns that arise should be addressed by the child’s physician. The athlete will also be coded in the dataset as having a potential concussion.

B.3. Methods to Maximize Response Rates and Deal with No Response

The approach to ensuring the highest possible retention of the athlete sample begins with a survey instrument designed with age-appropriate language and questions and tested in prior work with the target population. The survey instruments used in the current study are substantially similar to those used in the Lystedt Law study to successfully collect data from a similar target population (see survey samples in Attachments C through H). The response rates from that study are described in section B.1 along with our expected response rates for the current study.

To assess the possible attrition bias from participating athletes who drop out of the study, we will conduct nonresponse analyses. Analysis will include an examination of differences in demographic characteristics, location, and stratum between respondents and nonrespondents. We will make other efforts to retain athletes in the sample who are not responding (including reminder phone calls, e-mails, and text messages). The characteristics and survey responses of nonrespondents will be compared with athletes who have provided weekly reports. Outcome analyses will be adjusted to account for any identified biases in our final sample.

The study will make use of several methods to increase response rates throughout the study period. First, eligible participants will be sent an advance notification alerting them to the study, describing USYSA’s support for the study, and encouraging participation. Research suggests that advance notification can be effective in increasing response rates [2].

Enrolled participants who do not complete the preseason survey by the deadline will receive a follow-up e-mail reminder. If they still have not responded following the e-mail, a follow-up phone call will be made. Research suggests that an automated phone call with a prerecorded message can be effective at boosting response rates [3]. If participants have not responded after the e-mail and automated phone message, respondents will be contacted by phone by a member of the research team. General reminders to complete the survey, without information on study participants, may also be posted to the USYSA Web site and Facebook pages, individual team Web sites and Facebook pages, Twitter accounts, and any other available outlets.

When athletes are assented or consented into the study and parents have consented creating a athlete-parent dyad, we will offer them the choice of an automated phone call, e-mail, or text message for weekly reminders. They will then receive an automated prompt on Sunday evenings using their method of choice. The prompt will direct them to call into the IVR system using a toll-free number, where they will answer a brief survey. Athletes and parents who do not complete a weekly report will be sent an e-mail reminder notification. They will then be followed up with by phone, if necessary. Because the study team has chosen not to provide incentives for study participation, resources can be directed at intensive follow-up throughout the study period. The pilot study conducted in Washington State obtained 15-to-20 percentage point increases in response rates to the weekly surveillance reports with follow-up. We believe that the resources available for follow-up in the current study can produce similar, if not higher, results.

The study will also make use of passive recruiting and reminders in an effort to increase response rates. The study team expects to post survey announcements flyers and notices on the USYSA Web site, the website of State soccer associations, and individual soccer club or team Web sites as well as the official Facebook pages of these groups. In addition, we anticipate using the above resources to post reminders to complete the online survey as well as weekly reminders to call the toll-free IVR number and complete the weekly surveillance report. The exact procedures we will use to achieve the desired high response rates will be determined in collaboration with USYSA leadership as well as the coaches, club presidents, and team parent liaisons of the selected team. Each team likely has an established way of communicating important information such as practice times or game cancellations. For instance, some teams may rely on e-mail, whereas others may use a team Web site or Facebook page to post important information. We will work with selected teams to identify and make use of existing communication channels.

Other factors in the study that are likely to contribute to a high response rate for enrolled participants include personalized communication, the use of identification numbers for study participants, and topic salience. Research suggests that personally addressing the survey participant can have a favorable effect on response rates [2]. Our survey invitation and reminders will address enrolled participants by first name wherever possible. In addition, research has found that using identification numbers provides participants with a sense that their identity is protected while also creating a sense of accountability to the research team because they can be contacted for reminders [2]. Our study will provide each participant with a unique identification code that will be used to complete the online survey and the weekly surveillance reports.

Finally, we expect that interest in the research topic will be high among the target population. It is likely that parents, coaches, and athletes will be aware of and interested in the issue of concussions in sports and thus will be motivated to participate in a study on the topic. Interest in the topic being studied can lead to higher response rates than if the topic were of little interest to the target population [2].

B.4. Test of Procedures or Methods to be Undertaken

The current study is based in large part on the Lystedt Law study undertaken by researchers at the University of Washington. The previous study similarly administered a preseason survey online, followed by weekly surveillance using an IVR system. The IVR system used will be the same one in place for the current study.

The survey instruments from the previous study were used successfully with a similar target population. The research team has made minor modifications to the preseason survey instruments as needed (e.g., changing references from “student” to “athlete”), and the weekly surveillance and injury follow-up procedures will be the same.

The original survey instrument was carefully developed by first conducting qualitative interviews using a standardized template with a random sample of key informants (high school coaches and athletic directors), then using these interviews to inform our coach survey. The athlete and parent surveys mirror the coach survey. The concussion knowledge questions included in the preseason surveys are excerpted from a standardized concussion knowledge form [4]. The concussion symptom checklist used in the weekly surveillance surveys is taken from the SCAT-2, a standardized method of evaluating injured athletes for concussion that can be used in athletes ages 10 and older.

The UW researchers successfully gathered the data needed for the Lystedt study using these methods. We anticipate similar success as we employ similar methods with the current study.

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

All instruments and procedures have been reviewed extensively by CDC. The following people have worked closely in developing the instrument and procedures that will be used, and they will be responsible for data analysis: Dr. Frederick Rivara (University of Washington), 206-744-9449; Dr. Ali Rowhani-Rahbar (University of Washington), 206-221-1602; Dr. Sara Chrisman (University of Washington), 206-484-2133; Dr. John Foster-Bey (CSR, Incorporated), 703-741-7131; and Dr. Z. Joan Wang (Avar Consulting), 301-977-6553.

Data collection and analysis will be undertaken by a team of contractors at CSR, Incorporated; Avar Consulting; and the University of Washington.

References

1. Rivara, F.P., Schiff, M.A., Chrisman, S.P., et al. (2014). The effect of coach education on reporting of concussions among high school athletes after passage of a concussion law. Am J Sports Med, 42(5), 1197–1203.

2. Anseel, F., Lievens, F., Schollaert, E., & Choragwicka, B. (2010). Response rates in organizational science, 1995–2008: A meta-analytic review and guidelines for survey researchers. J Bus Psychol, 25, 335–349.

3. McCarthy, J.S. (2007). Pre-Recorded Telephone Messages as an Alternative to Follow-Up Survey Reminder Postcards: The Effectiveness of Autodialer Software Used During the Pennsylvania County Estimates Survey. USDA RDD Research Report Number RDD-07-04.

4. Rosenbaum, A.M., & Arnett, P.A. (2010). The development of a survey to examine knowledge about and attitudes toward concussion in high-school students. J Clin Exp Neuropsychol, 32(1), 44–55.

2