Request for OMB Review
part b: Collections of Information Employing Statistical Methods
Program Assessment and Evaluations for NIEHS – Asthma Research
October 5, 2007
Project Officer Jerry Phelps
Division of Extramural Research and Training
National Institute of Environmental Health Sciences
P.O. Box 12233, MD ED-21
111 T.W. Alexander Drive,
Research Triangle Park, NC 27709
Phone: (919) 541- 4259
Fax: (919) 541-4937
Email: phelps@niehs.nih.gov
Table of Contents
Page
B.1. Respondent Universe and Sampling Methods 3
B.2. Procedures for the Collection of Information 3
B.3. Methods to Maximize Response Rates and Deal with Nonresponse 6
B.4. Tests of Procedures or Methods to be Undertaken 7
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 8
List of Attachments 9
B.1. Respondent Universe and Sampling Methods
The study population will include all principal investigators receiving funding for asthma-related grants between 1975 and 2005 from NIEHS and selected comparison agencies. The sampling frame will be all principal investigators (from the selected agencies) with grants in the NIH and EPA grants databases that report the term ‘Asthma’ in either grant title or abstract. Our calculation of 1550 respondents for the survey is based on a preliminary analysis of the NIH and EPA grants databases from 1975 to 2005.
To minimize possible bias from nonresponse and to maximize statistical power, the study aims to achieve a response rate of at least 75%. In surveying the universe of eligible principal investigators, the only source of sampling error is nonresponse. Therefore, our estimated standard error includes the following finite population correction factor:
Standard error=SQRT ((nonresponse rate)*(standard deviation)2/(no. of surveys in analysis))
B.2. Procedures for the Collection of Information
This section describes data information collection procedures. The discussion is divided into two subsections: (1) data collection procedures and (2) quality control procedures.
An initial hardcopy letter on NIEHS letterhead will be sent to all survey respondents alerting them to the survey and noting that an email will come from Battelle asking them to complete a web-based survey related to asthma research. Returned letters will be tracked.
An initial email will then be sent to the respondents identified through the NIH and EPA grants databases inviting them to participate in the survey, explaining the study, and obtaining informed consent. The email will provide the name and toll-free number of a staff member to call with questions about the study. The letter will also include the name and telephone number of a person to call with questions regarding Human Subjects protection. There will be web and paper options for completing the survey. The initial email will provide a link and secure login to complete the survey by web. The initial email will also include an email address and telephone number to request a paper version of the survey.
A paper copy of the survey with a postage-paid return envelope will be sent via express mail to all those respondents who request a paper version or for whom we do not have a valid email address. Within two weeks of this mailing, a mailed reminder postcard will be sent to all non-respondents. After tracing and verifying bounced-back emails, and after three email follow-up reminders, those respondents who have not yet completed the survey via the web will also be sent a paper copy.
Survey email invitations will be sent continuously over a one-month period. Battelle will track all returned surveys in the computer system upon receipt. Within one week of the initial email, a reminder email will be sent to each respondent to encourage survey completion. Two subsequent email reminders will be sent at 4-day intervals if the survey has not yet been completed. The email (Attachment 2-3) will include a toll-free number that can be called if a respondent has any questions about completing the survey or needs to have a copy of the survey mailed. Within two weeks of sending the final email reminder, a mailing of the survey packet via express mail will be sent to non-respondents.
Within two weeks of mailing the survey, one telephone follow-up call will be made to all remaining non-respondents to remind them to complete the survey. If the person is reached directly the interviewer will offer to conduct the interview over the telephone, otherwise a message will be left.
To maximize response rates, tracking of bad email addresses, mail addresses, and telephone numbers will be conducted and new information will be sought if possible.
Beginning with study initiation and continuing through all phases of data collection and analysis, steps will be taken to ensure that the data collected are of the highest quality possible. All project staff will be trained to understand the purpose, sponsorship, background, objectives, and importance of the project, as well as their specific role and activities on the study. In training project staff, Battelle will emphasize the steps that will be taken to protect the confidentiality of the data that are collected. Completed hardcopy survey questionnaires will be stored in locked file cabinets. All project files will be password protected and access to the files will be limited to authorized project staff. Surveys entered online will be password protected and will not allow access once the respondent has completed the survey. The web survey will be hosted on a secure server protected with a Secure Sockets Layer (SSL) certificate and 128-bit encryption, the strongest online data encryption protection available.
A management information system will be developed to monitor data collection activities. The database will contain the dates of all email, mail, and telephone contacts with respondents and be used to track the most up-to-date contact information. The web survey automatically tracks started and completed questionnaires, and project staff can see such information in real time. The management information system and web survey completion reports will also be used to generate daily reports summarizing the status of data collection activity through the data collection period.
An emphasis on quality will continue with data editing and data entry. A web based survey option will be encouraged in order to provide faster survey response time and availability of data as well as high quality data since control checks are built in. A mailed hard copy questionnaire will also be available. The hard copy will also be sent to those for whom email addresses are unavailable or as part of the follow-up protocol to increase response rates. To reduce the possibility of data entry errors, scannable hard copy survey forms will be used. These forms can be filled out on paper by the respondent, but data entry is automated when the forms are returned and scanned in. Battelle programmers will develop quality control checks for the scannable surveys to ensure data quality.
B.3. Methods to Maximize Response Rates and Deal with Nonresponse
Over the past 20 years, Battelle has developed procedures that have been successfully used to achieve response rates of 70 to 80% to surveys of individuals. Multiple methods studies, reviews, and meta-analyses have been conducted to determine which factors lead to an increase in response rates in mail surveys. Preliminary notification, multiple follow-ups with respondents, personalization techniques, sponsorship or endorsement, use of express mail, and shorter questionnaires have shown positive effects on response rates.
Response rates will be reported at each stage, i.e. the response rate from the initial and subsequent emails, from the initial and second mailing, the additional response rate following the postcard and telephone reminders. Once data collection has been completed, we will conduct a non-response analysis and adjust for non-response by weighting the survey data.
B.4. Tests of Procedures or Methods to be Undertaken
The draft survey questionnaire was developed with input from nine NIEHS principal investigators currently receiving either research or training funding related to asthma. During informational interviews with each investigator, we sought the following information from the informational interviews:
Ability to answer questions
Determination of length of survey
These nine principal investigators were asked to review the draft survey. Four were asked to review a hardcopy draft for content, clarity of questions, ease of response, and relevance of individual questions. The pretest was also used to obtain an estimate of respondent burden. Minor modifications to the survey content and format resulted from the review. Experienced survey operations staff will format the survey questionnaire for online ease of completion, as well as to facilitate coding and data entry. The other five will be asked to test the web version of the survey to ensure that the online data collection is working as planned.
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The primary contact at NIEHS for this data collection activity is Jerry Phelps, Division of Extramural Research and Training, National Institute of Environmental Health Sciences. His telephone number is (919) 541- 4259 and his email is phelps@niehs.nih.gov.
Researchers at Battelle Centers for Public Health Research and Evaluation (CPHRE) will collect the survey data and perform data analysis, in consultation with NIEHS. Carlyn Orians has overall technical responsibility for the study at Battelle and led the Battelle effort to design the survey and protocol. Ms. Orians will direct the overall data collection and analysis effort. She will also be responsible for writing the project reports.
Carlyn Orians, MA
Work Assignment 3 Leader
Battelle Centers for Public Health Research and Evaluation
(206) 528-3320
Email: orians@battelle.org
Other personnel providing statistical consultation included:
Jennifer Cohen, PhD
Battelle
(206) 528-3116
Email: cohenj@battelle.org
Other staff involved in the design of the protocol and data collection instruments are:
Howard Fishbein, PhD Shyanika Wijesinha Rose, MA
Battelle Task Leader Battelle Deputy Task Leader
(703) 248-1647 (919) 544-3717
Email: fishbeinh@battelle.org Email: rosesw@battelle.org
Ed Liebow, PhD Jennifer Cohen, PhD
Battelle WA2 Leader Battelle WA3 staff member
(206) 528-3105 (206) 528-3116
Email: liebowe@battelle.org Email: cohenj@battelle.org
List of Attachments
Attachment 1: Survey Instrument 1-1
Attachment 2: Introductory and Follow-up Letters 2-1
Attachment 3: Institutional Review Board Approval 3-1
Attachment 4: Table Shells for Analysis 4-1
| File Type | application/msword |
| Author | Battelle |
| Last Modified By | Battelle |
| File Modified | 2007-10-09 |
| File Created | 2007-10-08 |