Supporting Statement B for
Feedback Survey for the Brain Disorders in the Developing World Program of the John E. Fogarty International Center (FIC)
Date: May 31, 2013
Name: Rachel Sturke, PhD
Address: Fogarty International Center, 31 Center Drive - MSC 2220, Bethesda, MD 20892-2220
Telephone: 301.480.6025
Fax:
Email: sturkerachel@mail.nih.gov
B.1 Respondent Universe and Sampling Methods 4
B. 2. Procedures for the Collection of Information 5
B.3. Methods to Maximize Response Rates 6
B.4. Test of Procedures or Methods to be Undertaken 7
B.5. Individual Consulted on Statistical Aspects and Individual Collecting and/or Analyzing Data 7
List of attachments
Attachment 1: Text of welcome page and informed consent statement for awardee survey
Attachment 2: Estimate of hourly rate for foreign researchers
Attachment 3: Invitation to participate in the awardee survey
Attachment 4: Awardee survey instrument (US/HIC version)
Attachment 5: Awardee survey instrument (LMIC version)
Attachment 6: Reminder email for survey non-respondents
The John E. Fogarty International Center (FIC) proposes to conduct a web-based census survey of awardees of the Brain Disorders in the Developing World program. The universe of potential respondents consists of the primary collaborators associated with each of the 114 funded projects (as distinct from the 148 individual awards; see footnote).1 This typically includes two individuals per project: the principal investigator (usually based in the US or another high-income country (HIC)) and his or her primary foreign collaborator (usually based in a low- or middle-income country (LMIC)). However, there are a small number of projects where more than two individuals have participated as a primary collaborator (often due to leadership changes made between the R21 and R01 stages). There are also six individuals who have been primary collaborators on two distinct projects each; these individuals will be asked to complete the survey questionnaire separately for each project. The survey population therefore includes 262 individual-project entities, of which 123 are US or HIC-based and 139 are LMIC-based.
The survey population includes both the HIC- and LMIC-based collaborators for each project for several reasons. The most important reason is that it is anticipated each collaborator will have different information and different perspectives to contribute regarding most of the survey topics. This includes the details of various process issues (such as grant and contract management at each participating institution), research outputs and outcomes (including the ultimate fate of the research project, which may have been carried forward by either collaborator or both), and capacity-building outcomes (primarily at the LMIC institution, but the HIC collaborator’s perceptions are also of interest since the program design skews the power dynamics in the collaboration towards the PI). A secondary reason is that FIC’s past experience with awardee surveys has shown that HIC-based collaborators are slightly more likely to respond at all and much more likely to respond substantively to open-ended questions; therefore, asking all questions to both groups greatly enhances the likelihood that at least one member of each project team will make a substantive response to every question.
Sampling cannot be used within these groups because a key goal of the study is to exhaustively document research and capacity-building outcomes, especially whether and how research projects initiated with Brain Disorders funding have been successfully converted into sustainable research programs (by either or both collaborators) after the initial period of support. Given the general scarcity of funding opportunities for brain disorders research in the LMIC context, it is anticipated that the proportion of such projects will be relatively small. Sampling within populations that are already relatively small would result in a high probability of missing successful cases as well as potentially informative productive failures.
Estimates of the respondent universe for each category and anticipated number of participants are summarized in Table B.1.1.
Table B.1.1. Estimated Respondent Universe and Use of Sampling Methods by Category
Data Collection Instrument |
Estimated Size of Respondent Universe |
Number of Respondents Invited to Participate |
Expected Response Rate |
Expected Number of Responses |
Sampling Methods Used to Reduce Burden? |
LMIC participants |
139 |
139 |
80% |
112 |
No |
US/HIC participants |
123 |
123 |
80% |
99 |
No |
Principal investigators and collaborators will be invited to participate in a web-based survey via email containing a personalized link to the survey website (Attachment 3). The survey will be implemented via Qualtrics, a commercial software and web-hosting service for web-based surveys. The instrument will consist of 16 questions, of which most are multiple choice (designed to produce semi-quantitative data) and a few are open-ended. Question topics include program outputs and outcomes, key program management issues, perceptions of the program as a whole, and suggestions for improvement. Slightly different versions of the survey instruments will be used for HIC and LMIC participants in order to simplify the language (e.g. referring to the foreign site as “your home institution” vs. “your collaborator’s home institution”), but the substance of the questions will essentially be the same (Attachments 4 and 5). The survey will be open for approximately one month.
Responses will be downloaded from the Qualtrics website for tabulation and analysis. For semi-quantitative data, descriptive statistics will be calculated for each group as appropriate. Qualitative data from responses to open-ended questions will be coded and interpreted as described below for the interview data. Responses to questions about the characteristics and/or perspectives of individual respondents will be tabulated separately for US and LMIC-based respondents, and comparisons may be made as appropriate. For questions about the quantifiable outputs or ultimate fate of a specific funded project, all responses regarding a project will be combined, and descriptive statistics will be calculated for the set of projects as a whole.
Survey response rates will be calculated as the number of completed questionnaires divided by the number of invited participants. Participation will be monitored closely, and brief reminder emails will be sent to non-respondents on a weekly basis (Attachment 6). Based on prior experience with awardee surveys at FIC, we anticipate an 80% response rate overall, with a slightly better response rate from the US/HIC group than the LMIC group. Furthermore, we do expect to have responses from at least one collaborator for all of the funded projects. If there are projects for which neither collaborator responds, the contractor will work closely with the FIC program officer to make personal contact with the project team (via telephone if possible, otherwise via personalized email) in order to facilitate at least one response.
As the respondent group for each data collection instrument is relatively small, a formal pretest of the instruments will not be undertaken before they are fielded. However, all instruments have been reviewed extensively by program staff at FIC with detailed knowledge of the program as well as by experienced program evaluators employed by the contractor.
The design for this study was developed primarily by the Science and Technology Policy Institute (STPI), a contractor to FIC. Several individuals at FIC also participated in the development of the study plan. STPI will have primary responsibility for data collection and analysis. The lead investigator’s contact information is:
Christina Viola Srivastava
Research Associate
Science and Technology Policy Institute
Telephone: +1 617 721 9055
E-mail: cviola@ida.org
1 As noted in Supporting Statement Part A, there have been 148 unique Brain Disorders awards to date, but 37 of them are R01 awards that continue or expand a project that was initiated with an R21 award, so there have been only 114 unique funded projects.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | NDPA Proc Eval 30 day Fed Reg Supporting Statement |
| Author | STPI |
| File Modified | 0000-00-00 |
| File Created | 2021-01-29 |