Supporting Statement B for
OMB Clearance Request
National Heart, Lung, and Blood Institute
“Survey
of NHLBI Constituents’ Health Information Needs and
Preferred
Formats”
(NHLBI, NIH)
May 20, 2009
Name: Ann M. Taubenheim, Ph.D., M.S.N.
National Heart, Lung, and Blood Institute
Office of Communications and Legislative Activities
Address: 31 Center Drive, Building 31, Room 4A10
Bethesda, MD 21045
Telephone: 301–496–4236
Fax: 301–480–4907
E-mail: taubenha@nhlbi.nih.gov
B. Collections of Information Employing Statistical Methods 1
B.1. Respondent Universe and Sampling Methods 1
B.2. Procedures for Collection of Information 2
B.3. Methods To Maximize Response Rates and Deal With Nonresponse 3
B.4. Test of Procedures or Methods To Be Undertaken 4
B.5. Individuals Consulted on Statistical
Aspects and Individuals Collecting
and/or Analyzing Data 5
The National Heart, Lung, and Blood Institute (NHLBI) is requesting clearance to collect data with a survey for the purpose of creating a 3-year Strategic Plan (the “NHLBI Strategic Plan for the Development of Education Materials for Health Professionals and the Public”) for developing new health information materials for NHLBI constituents and revising materials currently in the Institute’s portfolio.
The NHLBI wishes to use a Web-based survey for this data collection. We will invite customers who have ordered NHLBI materials during the past 3 years from the NHLBI Health Information Center (NHLBI HIC) to participate in the survey.
The most effective method for soliciting respondents is to use a series of e-mail invitations that contain the live URL for the online survey. E-mail addresses are available for most NHLBI customers. By using customer information not more than 3 years old, we should reduce the number of e-mails addresses that are no longer in service. However, we will allow for non-operative e-mail addresses in determining the anticipated response rate to be used to calculate the total number of invitees to participate in the survey.
The total population of customers who ordered NHLBI materials from September 1, 2005, to August 31, 2008 (3-year period), for whom e-mail addresses are available is 35,724. This information is captured by NHLBI contractor American Institutes for Research (AIR) in its Great Plains management system. NHLBI HIC staff enter manually orders into Great Plains from customers contacting the NHLBI HIC by telephone, e-mail, postal mail, and fax. Customers also can place orders directly via the NHLBI Online Catalog at http://emall.nhlbihin.net/default.asp.
We based the estimates of the distribution of the type of customer and volume of orders on the population of customers who ordered NHLBI materials from September 1, 2005, to August 31, 2008. The actual sample will comprise NHLBI materials requesters and customers for the 3-year period immediately preceding data collection.
Information on the distribution of customers by type—patient, health professional, private sector organization, public sector organization—is collected by the NHLBI wherever possible. The trend for the distribution of customers placing orders, based on data collected since 1999, is as follows:
Customer Type |
Percentage |
n |
General Public |
43.9% |
15,675 |
Private Companies |
16.5% |
5,894 |
Public Sector Groups |
10.6% |
3,787 |
Health Professionals |
29.0% |
10,367 |
The NHLBI believes that inviting the total population of 35,724 customers with e-mail addresses is a valid method to obtain meaningful and representative data for the entire population of NHLBI HIC customers.
A sample of 2,450 respondents will
produce 99 percent confidence intervals of no more than
+/- 2.5
percent for percentage estimates. (This estimate employs a finite
population correction.)
For this survey, we will use simple random sampling (no stratification) of the total population of NHLBI HIC customers with a computer-based random sample generating software application.
Although every measure possible will be taken to maximize response rates (see next section), the NHLBI estimates conservatively that it will achieve a response rate of between 10 percent and 12 percent of the total population. Our estimate takes into account nonresponses due to bad or discontinued e-mail addresses, e-mail security measures, people who do not open e-mails, and people who choose not to complete the survey or only partially complete the survey, as defined by not answering any of the substantive questions (questions 5–13).
Based on an anticipated response rate of 10 percent, the original sample size needed to generate a random sample of 2,450 respondents is: 2,450/.1 = 24,500.
We will use a Web-based survey (WebSurveyor software) that can be completed online by respondents. We will invite the customers in the sample population to participate in the survey by e-mail. This e-mail will contain a link to the online survey. Respondents will log in using their e-mail address as their password. The WebSurveyor software will check the e-mail address against its list of e-mail addresses in the sample, thus preventing duplicate or multiple survey completion by the same individual. Respondents will be able to save, exit, and return to complete partially finished surveys.
WebSurveyor will check all responses for completion and accuracy, registering the data only for those completed surveys that pass the system’s requirements.
Through the use of built-in skip logic, respondents will be asked questions relevant to their customer type—patient/general public, health professional, etc.—based on their self-assessment of their background (see question 2 on survey instrument). This means that questions about materials geared primarily for health professionals (clinical practice guidelines, etc.) will be answered only by respondents who have designated themselves as health professionals. Intelligence built into the application will prevent multiple responses when single responses are required, allow multiple responses where applicable, guide respondents to the next step, provide online help, and allow respondents to save and complete the survey later.
For multiple-choice questions with more than six possible responses, randomization features built into WebSurveyor will reorder the choices to minimize the impact of order effects.
We do not anticipate any unusual problems requiring specialized sampling procedures.
We are proposing a 3-year data collection cycle to reduce burden and ensure that data on evolving trends are captured in an effective manner.
Survey response data will be collected by the WebSurveyor software package and will be made available to data analysts AIR, as well as relevant NHLBI Office of Communications and Legislative Activities (OCLA) staff.
This survey will employ best practices for Web-based surveys as outlined in the following publications:
Mail and Internet Surveys: The Tailored Design Approach, 2007 Update. Donald Dillman. New York: Jon Wiley and Sons. 2007.
E-Mail Survey Response Rates: A Review. Kim Sheehan. Journal of Computer-Mediated Communication. Vol. 6, Issue 2. 2001.
Introduction to Survey Methodology and Design. James K. Doyle. Chapter 10 from Handbook for IQP Advisors and Students, by Douglas W. Woods. Worcester, MA: Worcester Polytechnic Institute. 2006.
Using E-Mail to Survey Internet Users in the United States: Methodology and Assessment. Kim Sheehan. Journal of Computer-Mediated Communication. Vol. 4, Issue 3. 1999.
The NHLBI estimates conservatively that it will achieve a response rate of between 10 percent and 12 percent of the total population. To achieve the highest possible response rates, this survey will use “repeated, personalized attempts” to invite NHLBI customers to complete the survey.
We will send an initial e-mail to all members of the sample population explaining the survey (purpose, confidentiality, about the NHLBI, why they are being invited). This e-mail will not include the URL for the survey (see both Sheehan articles). The initial communication will inform potential respondents that they will receive another e-mail within a specified timeframe, which will contain the URL to the survey and instructions for completing it.
The initial e-mail will allow the sample population members to opt in or opt out of the survey. If an individual chooses to opt out, he/she will not receive any further communications about the survey from the NHLBI. The NHLBI will use Lyris e-mail software to send the e-mails and track respondents who opt in and opt out. This information will be transferred to the WebSurveyor software prior to sending out the URL to build the database of valid e-mail addresses for respondents.
This best practice allows the NHLBI to “clean” its sample of customers without viable e-mail addresses and make any necessary adjustments to the random sample. It also will provide respondents with the option to tell the NHLBI if they would prefer the survey URL be sent to a different e-mail address.
We will send a second, personalized e-mail to all e-mail addresses in the final sample population. This e-mail will contain some of the same background information as the initial invitation, along with the URL and instructions for completing the survey online. Respondents will have the option to go directly to the survey and bypass any contextual information if they so desire.
The NHLBI will send the survey respondents a thank-you e-mail after they complete the survey, with an opt-in option for receiving a summary of survey results.
We will send nonrespondents (who have not chosen to opt-out of the survey) a second personalized e-mail invitation with the survey URL.
We will send to those still not responding (and who have not opted out) a third and final personalized e-mail invitation with the survey URL.
To ensure that respondents are receptive to completing the survey once they’ve gone to the URL, the survey itself has been designed to eliminate as many obstacles as possible to completing it.
The survey has been pretested by groups of eight people to test for clarity and ease of response. Adjustments were made on the basis of feedback collected at the pretests.
The number of questions asked ranges from 12 to 13, depending on the self-assessed background of the respondent.
The use of skip logic maximizes the chances that questions will be meaningful and relevant to respondents based on their self-reported background characteristics.
All questions geared at collecting general (nonpersonal) demographic data on age, background, and occupation are asked at the beginning of the survey.
Questions are brief and written in language at a ninth-grade level.
Clear instructions are built into the survey instrument and are supplemented by intelligence built into the WebSurveyor tool.
Opportunities for writing in answers under an “Other” option are provided throughout the survey to provide respondents with the ability to state their preferences and needs clearly where listed options do not match their ideas.
The survey instrument has been pilot-tested with eight individuals who have experience using NHLBI health information materials, but no prior knowledge or exposure to the survey instrument, to ensure that the questions, instructions, and response options are clear and straightforward. The NHLBI also has received Institutional Review Board (IRB) approval for the study and the survey instrument.
The following individuals were consulted on the statistical aspects of the proposed survey:
Roger Levine, Ph.D., Managing Research Scientist, American Institutes for Research, 650–843–8160
Christine Paulsen, Ph.D., Principal Research Scientist, American Institutes for Research, 978–302–5543
The following individuals will direct the collection and analysis of data for this survey:
Eileen Smith, Marketing Director, American Institutes for Research, 301–592–3330
Lawrence Thomas, Project Director, American Institutes for Research, 240–629–3232
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | American Institutes for Research |
| File Modified | 0000-00-00 |
| File Created | 2021-02-03 |