RTI will conduct 55 qualitative interviews with a sample of 34 clinicians and all 21 RBH managers. (includes RBH management team and senior administrative staff). Exhibit 7 presents the sample statistics for each group. With assistance from RBH management, RTI will obtain complete lists of the names of clinicians who received training from IMC, according to type of clinician. These lists will serve as sampling frames for the three clinician groups who will participate in the qualitative interviews. These frames will be used to select a 20% sample from each group. We do not propose to stratify within clinician groups because the variables needed to do this are either unavailable to the evaluation team or unreliable, as determined by a two person team from RTI that traveled to Kabul in December 2007.1 Breaking the clinician interviews into these respondent groups is appropriate because of differences in the level of training each group received from IMC, and the markedly different roles they play in providing patient care and clinical training at RBH. These interviews will occur over a period of four (4) weeks, with prior agreement by the RBH Executive Director to maximize staff availability for scheduling and administering the qualitative interviews. In the non-census sampling frames, if one staff member is unavailable the next clinician on the list will be selected for interview. Both of these approaches will help to maximize response rate.
RTI will conduct qualitative interviews with a census of RBH management staff because we expect this respondent group to have substantial heterogeneity. The source of this heterogeneity is due to the managers coming from diverse occupations and functional areas. Each manager heads a different functional area, such as administration, blood bank, anesthesiology, lab, and pharmacy.
Exhibit 1–Qualitative Interview Sample Statistics
Type of Respondent |
Population |
Sample |
Expected Response Rate |
Total expected Respondents |
Midwives – Frame A |
75 |
15 |
95% |
15 |
Residents– Frame B |
56 |
11 |
95% |
11 |
Attending Physicians – Frame C |
16 |
8 |
95% |
8 |
Management Staff |
21 |
21 |
95% |
21 |
Except as noted above, gathering qualitative information does not require an entire census. Experience of qualitative researchers shows that at some number of qualitative interviews saturation occurs.2 Saturation is the state where little additional information is yielded by additional interviews. The number of interviews required to reach saturation depends on various factors such as the homogeneity of the respondents and the research questions of interest. Saturation occurs faster in more homogeneous groups. On factors of interest to this evaluation, we expect that individuals within a group are more alike than individuals across groups. As noted earlier, however, data are lacking to stratify within groups. We expect that saturation will be reached with the proposed sample for each clinician group, and note that the final sample size for each group could be lower than presented in Exhibit 7 if saturation is achieved prior to reaching the planned sample size.
RTI will administer the modified SBMAs to a census of clinicians who received training from IMC (see Exhibit 8). The decision to take a census versus selecting a sample is based on prior use of SBMAs by IMC and the needs of the evaluation. In prior IMC SBMAs, IMC achieved a response rate of 94%. RTI expects to compare the findings from this round of SBMAs with those from previous rounds. Prior rounds of the SBMA included a census of clinical staff. A census ensures that the findings are fully representative of the respondent groups. Furthermore, the overall number of clinicians in each respondent group (OB/GYN residents and midwives) would be too small to select a sample for each respondent group that would give sufficient power to detect a reasonable amount of change over time. A census of individuals avoids this problem, because results from the different rounds of data collection can be compared directly, without concerns about power or statistical tests.
Exhibit 2–SBMAs Sample
Type of Respondent |
Population |
Expected Response Rate |
Midwives |
75 |
95% |
1st Year Residents |
31 |
95% |
2nd Year Residents |
8 |
95% |
3rd Year Residents |
9 |
95% |
4th Year Residents |
8 |
95% |
Qualitative Interviews. Regarding selection of clinicians to participate in the qualitative interviews, RTI will select systematic samples from sample frames A (midwives), B (residents), and C (attending physicians) (see Exhibit 7). As noted under Section B.1, the sample frames will consist of an alphabetical listing of the names of clinicians who received IMC training that are provided to RTI by RBH management. If a selected participant declines participation for the qualitative interview, we will select a replacement respondent from the complete sample. If the person is unavailable, then the immediate next person on the list will be chosen to replace the selected participant. Each participant will be interviewed only once. The RTI team (RTI and D3 Systems/ACSOR staff) will conduct the qualitative interviews, with RTI staff obtaining informed consent and leading the interviews, and D3 Systems/ACSOR staff providing translation and transcription services. The team will record the interview to assist with note keeping and translation. The interview guide for clinicians consists of a set of common questions that will be asked of all clinicians interviewed, as well as a set of questions that are specific to each group. To minimize data collection and data entry errors, RTI will train members of the evaluation team on the use of these interview instruments, as well as prepare a Manual of Operations, which delineates the procedures for collecting and managing data collected for the evaluation. RTI will not attempt to contact interview participants after the interview is complete. Any clarification needed will be asked by the RTI staff leading the interview during the interview and before the interview concludes. No personal identifying information will be recorded on the interview notes.
All data from the completed qualitative interviews will be entered into Microsoft WORD and coded using NVivo. RTI staff will enter the handwritten notes into Microsoft WORD for each interview. These are reviewed for consistency with the handwritten notes, and once these are determined to be accurate, they are then converted to RTF and coded in NVivo.
Standards-Based Management Assessments. For the SBMA written exams and case studies, RTI, an ex-patriate OB/GYN physician, two local Kabul based OB/GYN physicians, and staff from D3 Systems/ACSOR will administer these components. RTI and the ex-pat OB/GYN will take a lead role in this data collection activity. All information collected, from the written exam responses, the observation notes, and the case study responses will be keyed into a D3 Systems/ACSOR electronic data collection system in Kabul. Prior to data entry, the de-identified data from the exams and case studies will be translated into English, with each translation verified by RTI and D3 Systems/ACSOR staff. Ten percent of the entries will be randomly selected for keying a second time (double entry). The accuracy of the data entry process will be verified by comparing the data from the first entry with the data from the second entry. The double keying verification process will allow researchers to report the rate of accuracy to the Project Director. To minimize data collection and data entry errors, RTI will train members of the evaluation team on the use of the SBM assessment, as well as prepare a Manual of Operations, which delineates the procedures for collecting and managing data collected for the evaluation. The data collected will be entered on a weekly basis, and is anticipated to be completed within two weeks of the close of data collection.
RTI and D3 Systems/ACSOR records retention policy does not conflict with the HHS OS records disposition schedule (Appendix A, Chapter 300).
RTI expects to achieve at least a 95% response rate for interviews and SBMAs for several reasons, including high levels of interest and support for these projects from the RBH Executive Director and other RBH staff, close coordination with RBH management to plan and schedule data collection to maximize response, in-person data collection, and use of instruments that have been piloted. These two projects have provided important financial and technical support to RBH. Since their implementation in 2003, OGHA, CURE and IMC have built a strong working relationship with the RBH Executive Director and staff.
The RTI team will work closely with RBH management and the RBH Executive Director to inform relevant RBH staff about the program evaluation and to stress the importance of their participation. These discussions will occur either at separate meetings, or as a part of the daily clinical staff meeting. The RTI team will also work with RBH management to schedule data collection activities at times that are sanctioned by the RBH Executive Director, are convenient for the participants, do not disrupt patient care, and do not occur on local holidays. Finally, to ensure that their content and structure is user-friendly, all data collection instruments will be pilot tested (see Section B.4), and revised based on feedback from pilot subjects. The self-administered SBMA written exams are tailored to the respondent group and have no skip patterns. Copies of the interview topic guides, self-administered written exams, and interviewer-administered case studies are presented in Appendix D through Appendix J.
During the initial OMB review period and after RTI has received approval from Institutional Review Boards at RTI and the Afghanistan Public Health Institute, RTI will pilot test the management qualitative interview guide with six individuals (in a mixture of English and Dari) and the clinician qualitative interview guide with six individuals. Pilot participants are from the D3 Systems/ACSOR staff and the two local OB/GYN physicians participating in the evaluation. This will be an opportunistic sample based on the timing of the pilot test and availability of these staff. RTI does not plan to pilot the SBMAs because an expanded version has been used at RBH to measure clinicians’ knowledge, skills, and decision competency three times since September 2006.
The following individuals contributed to the questionnaire and study design and will be involved in the interpretation and analysis of findings.
Dr. James Lea
|
Phone: 919-962-6801 Email: james.lea@gmail.com
|
Dr. Catherine Elkins
|
Phone: 919-541-8898 Email: celkins@rti.org
|
Niamh Darcy
|
Phone: 919-485-2610 Email: ndarcy@rti.org
|
Dr. Rod Knight
|
Phone: 919-918-7678 Email: rodjknight@aol.com |
Karl Feld
|
Phone: 919-601-7060 Email: Karl.Feld@d3systems.com |
Matt Warshaw
|
Phone: 0799-328-714 Email: Matt.Warshaw@aol.com |
1 The RTI team found that (1) information on the performance, placement, and rotation of OB/GYN residents was maintained at the Ministry of Public Health (MoPH) and not at RBH; (2) information on the years of practice, tenure at RBH, and areas of expertise for RBH midwives is incomplete or unavailable due to weak record keeping practices; and (3) criteria and processes for selecting attending physicians to participate in the IMC-sponsored or for residents to rotate through RBH are unclear.
2 See Miles, M.B. and Huberman, A.M. (1994). Qualitative Data Analysis, 2nd Ed., Newbury Park, CA: Sage, pp. 10-12; Sandelowski, M. (1995). Sample size in qualitative research. Research in Nursing and Health. (18):179-183; Guest, G., Bunce, A. and Johnson, L. (2006). How many interviews are enough?, Field Methods 18(1): 59–82.
| File Type | application/msword |
| Author | DHHS |
| Last Modified By | DHHS |
| File Modified | 2008-07-09 |
| File Created | 2008-07-09 |