Supporting Statement B for:
NINR End-of-Life and Palliative Care Science Needs Assessment: Funding Source Questionnaire
National Institute of Nursing Research (NINR)
6701 Democracy Blvd, Suite 710/ Room 724
One Democracy Plaza
Bethesda, MD 20892
Telephone: 301-496-9601
Fax: 301-480-8260
E-mail: amanda.greene@nih.gov
Table of Contents
B. Statistical Methods 1
B.1 Respondent Universe and Sampling Methods 1
B.2 Procedures for the Collection of Information 1
B.3 Methods to Maximize Response Rates and Deal with Nonresponse 3
B.4 Test of Procedures or Methods to be Undertaken 3
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 4
Attachment 2: Invitation Letter
Attachment 3: Informed Consent
This supporting statement addresses data collection procedures related to the evaluation of the National Institute of Nursing Research (NINR) End-of Life and Palliative Care Science Needs Assessment: Funding Source Questionnaire. This section will describe in detail the various aspects of the questionnaire data collection methods.
Data will be collected from all authors who published an end-of-life and palliative care (EOL PC) research study between 1997 and 2010, when a funding source was not cited or the funding source information was unclear in the publication. The potential respondents will be identified from a review of published and publicly available data on EOL PC research during that time period. Funding source information on published studies is commonly included in all publications; however, such information is missing for some, particularly for studies conducted in earlier years1. A preliminary review of the literature on EOL PC research in the specified timeframe identified 2300 related publications for which a funding source was not cited or the information was unclear. All potential respondents will be surveyed, rather than a sample, in order to quantify the number of EOL PC research studies that did not receive grant funding.
Efforts will be made to obtain a list of potential participants with full contact information to maintain consistency of procedures and data quality. Available contact information on main authors often includes full name, organization affiliation and address, business e-mail address, and business telephone numbers. This author contact information is publically available and listed on their published EOL PC research study or listed on their institution’s public website. This information is necessary for maintaining contact with the respondents. This questionnaire is voluntary. A high response rate –80 percent - is expected because authors have published their EOL PC research in peer-reviewed journals, as such, the authors have a responsibility to their audience to provide information associated with their published studies. In addition, these authors are researchers who want their work and research support mechanisms to be recognized, which can influence their future research awards.
a. Questionnaire Procedures
Personal identity of potential respondents will be protected by assigning a confidential ID number to each response so that the respondent’s name and any other personal information will not be directly linked with their response. The identity of respondents will not be released. Identifying information of respondents will be stored separately from the questionnaire responses. All collected survey information will be analyzed and reported in aggregate.
Email invitation letter. An e-mail invitation letter to complete the questionnaire will be sent to the first authors of the identified EOL PC publications for whom a funding source is unknown or the funding source citation is unclear. (The invitation letter is in Attachment 2). The letter will indicate that the study is sponsored by the National Institute of Nursing Research of the National Institutes of Health (NIH). Specifically, the letter will:
Briefly describe the EOL PC Evaluation Needs Assessment, the purpose of the Funding Source Questionnaire, and the importance of respondents’ participation in the evaluation needs assessment.
Inform participants that completing the Funding Source Questionnaire online will take about 5 minutes.
Specify that the identity of respondents will not be released. Identifying information will be stored separately from the questionnaire responses. All collected information will be analyzed in aggregate.
Include a URL address that will take participants directly to a secure Website with the questionnaire.
Provide contact information in case respondents have any questions about the study.
Be sent from an NIH project email address (assigned to NOVA’s project manager, Dr. Lisbeth Jarama). E-mail replies and bounce-backs will be sent to this NIH email address.
Dr. Amanda Greene, NINR evaluation officer, will sign the email invitation letter.
Two weeks after the initial email announcement, a reminder e-mail (Attachment 2) will be sent to non-respondents. If one month after the first email, less than the desired percentage of potential participants have responded to the questionnaire, a second reminder follow-up email will be sent to encourage response. Potential participants will experience no burden if they choose not to respond.
Informed Consent. When potential participants access the secure, questionnaire Web site, they will be routed to the informed consent form. The consent form may be found in Attachment 3. In addition to being informed about the EOL PC evaluation needs assessment and the purpose of the Funding Source Questionnaire, potential participants will be provided with more details regarding their rights as participants in the study. Specifically, the consent form will:
Describe the process to ensure confidentiality of the information provided by participants.
Explain how participants’ identity and anonymity will be protected.
Explain that participation is voluntary and that respondents can stop participation at any time.
Describe how the data collected will be protected and safeguarded.
Provide contact information in case respondents have any questions about the study.
Potential participants will be prompted to accept or decline participation by selecting the appropriate button at the bottom of the electronic form. Only if participants click on the “I accept” button, they will be routed to the first question in the questionnaire.
Survey Data Collection. Respondents will be directed to a secure, web-based questionnaire. Respondents will be assigned a confidential ID number. Personal identifying information will be stored separately from responses. Responses to questions will be collected in an electronic database that can be analyzed using statistical software such as Excel, SPSS, STATA, or SAS. Data files will be kept in a secure environment and no one outside of this study will have access to them.
This is a one-time study collection.
b. Rationale for Surveying All Respondents of a Given Type
Information on funding sources of published studies is commonly included in the publication itself. However, in the case of some EOL PC studies this information is not included or the funding source information is unclear. This may be common prior to requirements to cite federal funding in public databases such as PubMed.
A preliminary review of the literature from 1997-2010 using key search terms related to EOL PC identified 2300 related publications for which a funding source was not cited or the information was unclear. All potential respondents will be surveyed, rather than a sample, in order to quantify the number of EOL PC research studies that did not receive grant funding.
Although the target respondent group consists of 2300 participants, the number of expected respondents is 1840. The number of expected respondents – 1840 participants – is based on a target response of 80 percent. A high response rate is expected because the authors have published their EOL PC research in journals, as such, the authors have a responsibility to their audience to provide information associated with their studies.
Several procedures will be implemented to maximize the response rate for the Funding Source Questionnaire. The questionnaire has been designed for maximum ease of administration, and the data collection protocol has been tailored to acknowledge respondents’ contributions. The presentation of the questionnaire has also been taken into consideration so that respondents can differentiate it from other mail and research requests.
The introductory letter or e-mail with the link to the questionnaire will indicate that it is sponsored by NINR. The e-mail will succinctly inform readers of the importance of the questionnaire, as well as procedures for maintaining the privacy of the respondents. The e-mail will be sent from an NIH address.
Online administration of the questionnaire is expected to greatly increase the ease of data collection for these sampled individuals. The strategy for email follow-up for those who have not responded to the e-mail request has been carefully designed. By following up consistently and persistently, NINR will demonstrate that it is committing time and energy to obtain the most valid data possible by obtaining the funding information from as many EOL PC researchers as possible.
Consistent with the response rate calculations approved by the American Association for Public Opinion Research (AAPOR), response rates for this study will be calculated as follows:
Number of Completed Questionnaires |
Number of Completed Questionnaires + Number of Nonrespondents |
Although the target respondent group consists of an estimated 2300 participants, the number of expected respondents is 1840. The number of expected respondents – 1840 participants – is based on a target response of 80 percent. A high response rate is expected because the authors have published their EOL PC research in journals and have a responsibility to their audience to provide information associated with their studies. In addition, these authors are researchers who want their work and research support mechanisms to be recognized, which can influence their future research awards.
The Funding Source Questionnaire was programmed for Web-based administration and tested by NOVA staff before pilot-testing. A pilot-test of the Funding Source Questionnaire using a preliminary instrument that contained draft items for the final Questionnaire was conducted to ensure that useful and relevant information was gathered. Main data collection aspects and evaluation procedures were also tested with the pilot-study. Information gathered from the pilot-test was summarized and used to improve data collection using the Funding Source Questionnaire.
The Pilot-test was conducted with 8 individuals. Two respondents reported receiving funding for their published study; six respondents reported not receiving funding for their published study. A cognitive testing interview was conducted with one of the respondents who received funding for the published study to inquire about her experience completing the Funding Source pilot test survey and related procedures. According to this respondent:
The online survey was appropriate in length (e.g., short), format and sequence of questions, and appearance (e.g., font size).
Survey instructions and questions were clear and response options appropriate.
Recalling information was often challenging as the study in question was completed a long time ago and the funding received came from several small grants.
The invitation process to complete the survey was appropriate; e-mail communications rather than phone calls are the best way to be contacted (Note: at the time of the interview, she had not retrieved the phone message we had left for her the previous week).
No other difficulties or recommendations regarding completion of the survey were reported.
NOVA worked together with the NINR Science Evaluation Officer and other designated NINR staff to implement needed refinements in evaluation procedures.
The contractor collecting and analyzing information for the EOL PC Needs Assessment will be NOVA Research Company. Responsibility for collecting and analyzing information obtained through the methodologies described above will rest with NOVA. All data collection and analysis will be performed in compliance with Office of Management and Budget, Privacy Act, and Protection of Human Subjects requirements. The EOL PC Needs Assessment project manager from NOVA Research Company is Dr. S. Lisbeth Jarama (office telephone: 301-986-1901; email: LJarama@novaresearch.com and email: jaramasl@mail.nih.gov).
1 Since 2008, NIH has required that all publications resulting from NIH funded studies cite the NIH grant number. Prior to this time, citation of funding was optional although some science journals required citation of funding.
| File Type | application/msword |
| File Title | TABLE OF CONTENTS |
| Author | Vivian Horovitch-Kelley |
| Last Modified By | OD/USER |
| File Modified | 2012-02-07 |
| File Created | 2012-02-07 |