Formative Research and Tool Development for the
Medical Monitoring Project: Case-Surveillance-Based
Sampling as an Alternate Sampling Method to Include HIV-Diagnosed People Both Receiving and Not Receiving HIV Care
Generic Information Collection request under 0920-0840
July 24, 2012
Supporting Statement
Part B
Contact:
Stanley Wei, MD, MPH
Medical Epidemiologist, Clinical Outcomes Team
Division of HIV/AIDS Prevention
Centers for Disease Control & Prevention
1600 Clifton Rd, NE, MS E-46
Phone (404) 639-4288
Fax (404) 639-8640
Swei1@cdc.gov
TABLE OF Contents
Section B Justification
1 Respondent Universe and Sampling Method
2 Procedures for the Collection of Information
3 Methods to Maximize Response Rates and Deal with Non-response
4 Tests of Procedures or Methods to be Undertaken
5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The respondent universe is HIV-diagnosed persons ≥ 18 years old, currently residing in one of five demonstration project areas who have been reported to the National HIV Surveillance System (NHSS), OMB Control No. 0920-0573 Exp. 1/31/2013: Adult and Pediatric Confidential HIV/AIDS Case Reports.
The proposed formative research project will explore the feasibility of using a new sampling frame and method for MMP—stratified sampling directly from NHSS. When MMP was designed, no sampling frame existed from which to select a probability sample representing HIV-diagnosed persons in the United States. Therefore, a facility-based multi-stage cluster sampling approach was employed. This sampling method excludes an important group—HIV-diagnosed persons who are not receiving care. Because it includes all HIV-diagnosed persons, both receiving and not receiving HIV care, NHSS could potentially serve in place of the facility-based sampling frame MMP currently employs.
If successful, stratified sampling using NHSS as a sampling frame could significantly reduce costs associated with the current complex sampling design, and increase the scope and usefulness of MMP by including HIV-diagnosed people who are not receiving care. In this demonstration project, we plan to oversample recently diagnosed patients in order to collect information critical for improving HIV testing and linkage to care services and enhancing HIV prevention interventions. As part of the demonstration project, selected health departments currently participating in MMP will draw a sample of eligible persons from local surveillance data, find and recruit them (i.e., screen them for eligibility and offer enrollment in MMP), and conduct interviews with and abstract the medical records of those who consent. The use of NHSS as a sampling frame for MMP eliminates the need for sampling of facilities, as patients will be sampled directly from NHSS. The data from this demonstration project will guide the future design of MMP.
Sampling Frame
Five demonstration areas have been selected by an open, competitive process from among the 23 current MMP project areas: Los Angeles, CA; Mississippi; New York City, NY; San Francisco, CA; and Washington state. The estimated number of persons available for selection in each of these areas is 52,870, 10,039, 112,605, 22,430, and 14,751, respectively, or a total of 212,695 persons. Selection of a sample of persons from NHSS will occur independently in each of these demonstration area strata.
For the first demonstration year, the sampling frame will include persons who:
are present in the NHSS database of the demonstration project area, i.e. reported as having been diagnosed with HIV infection;
meet the surveillance case definition for HIV infection;
have been diagnosed with HIV as of a reference date, hereafter referred to as the sampling date (the sampling date may be chosen as several months prior to the date on which the sample is actually drawn so as to allow for reporting delay); and
are ≥ 18 years old on the sampling date.
This deliberately broad sampling frame will allow an assessment of the feasibility of sampling from many subpopulations in the universe of persons reported to HIV case surveillance, inclusive of populations with differing levels of engagement in care and differing time since diagnosis. If feasible, the following additional inclusion criterion will be added:
currently residing in the project area.
It may be possible to add this inclusion criterion if most recent address is available for most cases reported to NHSS. If not, an alternate criterion, such as “diagnosed within the demonstration project area,” may be added. This decision will be made after an evaluation of available data.
Persons whose death is documented in NHSS records will be excluded from the sampling frame.
Drawing the sample
Two-hundred participants per year will be sampled in each demonstration area by stratified random sample. The HIV/AIDS epidemic was first described in 1981 and life expectancies for HIV patients on anti-retroviral therapy are approaching those in the general population. In the five CSBS project areas, 3% were diagnosed < 1 year ago, 16% were diagnosed between 1 and 5 years ago, and 81% were diagnosed ≥ 5 years ago. In a simple random sample, only 19% would be expected to have been diagnosed < 5 years ago. However, some of the most critical public health questions concern the younger and more recently diagnosed HIV-infected population.
Therefore, recently diagnosed persons will be oversampled. The exact sampling proportions will be influenced by a review of eHARS data from the selected demonstration areas. However, we anticipate using a stratification scheme and sample such that the proportions of the total sample are as follows:
• 10% diagnosed ≤ 1 year from the sampling date
• 40% diagnosed 1-4 years from the sampling date
• 50% diagnosed ≥ 5 years from the sampling date or among whom date of diagnosis is unknown
This plan will be confirmed pending a thorough review of NHSS data.
In order to determine a minimum sample size, the expected precision of estimates derived from the entire sample and from subpopulations were considered for different sample size options. It was determined that a sample size of 200 persons per project area or 1,000 persons overall would have both acceptable precision and feasibility based on estimates of precision.
In calculating these estimates, the impact of weighted data analysis on precision was taken into account. Weighted analysis is necessary because the use of stratified random sampling within project areas and adjustment for non-response bias cause unequal selection probabilities. Both unequal selection probabilities and correlation of observations within project areas mean that variance estimates will be larger than they would be for a simple random sample of the same size. This variance inflation is called design effect (df). A design effect of 2 is used in the calculations because that level of design effect is commonly encountered in national surveys.
The following table shows the expected precision of an estimate from these data, such as an estimate of the proportion of persons who identified finances as a barrier to receiving care. The confidence interval (CI) half-widths in the table are the maximum that would be expected for estimates based on sample sizes of 200 and 1,000 for project area and aggregated estimates, respectively.
The table shows the level of precision to be expected not only for estimates for the entire population (column 2), but also for subpopulations that comprise 50%, 33%, and 10% of the total population (column 3, 4, and 5 respectively).
|
CI half-width |
CI half-width |
CI half-width |
CI half-width |
N |
total population |
subpopn = 50% |
subpopn = 33% |
subpopn = 10% |
200 |
9.80% |
13.86% |
17.06% |
30.99% |
1,000 |
4.38% |
6.20% |
7.63% |
13.86% |
Expected Response Rate
The proposed formative research is designed to evaluate a new sampling method as a potential replacement for the current MMP sampling methodology. Current facility-based MMP methods require facility participation as a pre-requisite for patient participation. Facilities can sometimes be barriers to patient recruitment and thus affect overall response rates. The facility response rate for MMP was 76% in 2009 and 80% and 83% in 2010 and 2011, respectively. The use of the NHSS as a sampling frame would remove facilities as a potential barrier to patient recruitment. The response rate is expected to be the same as or better than the MMP response rate for diagnosed persons receiving HIV care. In 2010, 55% of eligible persons sampled for MMP were successfully interviewed.
The proposed project will test a new sampling methodology for the OMB-approved data collection—MMP (0920-0740, expires 5/31/2015). Patients will not be sampled from facilities, and therefore, recruitment will be unlike MMP in that it will not occur exclusively through providers. In most cases, recruitment will be through direct contact with participants or with contact through providers employed as a back-up if direct contact fails for participants with a known provider. Otherwise, data collection procedures, described below, will be exactly the same as for MMP.
All eligibility screening and interviews will be conducted by trained project staff. Participation in the project is voluntary. Respondents may refuse to participate at all or in part. Respondents may refuse to answer questions or stop participation at any time without penalty.
The National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention (NCHHSTP), CDC, has determined that MMP is not research and that it is a routine disease surveillance activity, with data being used for disease control program or policy purposes (Appendix H). Because NCHHSTP has determined that MMP is not research, it is not subject to human subjects regulations, including federal institutional review board (IRB) review and approval. All federal, state, and local MMP staff must adhere to the ethical principles and standards by respecting and protecting the privacy, confidentiality, and autonomy of participants to the maximum extent possible.
The CDC Clinical Outcomes Team, which manages MMP, has requested a non-research determination for the CSBS pilot. This application is currently under review.
CSBS project areas should follow state and/or local procedures to determine whether the demonstration project is subject to state and/or local human subject regulations. The need for state/local IRB review, and the IRB approval and renewal dates, if applicable, must be kept on file in every project area. Copies of this documentation should be provided to CDC on an annual basis.
Sampled persons will be offered enrollment primarily through staff-contact enrollment. However, some providers may prefer to contact the patient first and let them know they have been selected to participate. For direct contact by CSBS staff, potential participants will be initially contacted using letters or personal- and telephone-contact scripts developed using CDC templates (Attachments 9a and 9b).
Contact information for sampled persons being sought for recruitment will be obtained from NHSS. Prior to making phone contact, project areas may send information about CSBS by mail, although such mailings will refer in general to conduct of a health survey rather than specifically mention HIV. Local CSBS staff will use patient contact information to initiate phone contact with eligible persons to describe the project and offer enrollment. Difficult to locate or contact patients may be approached at their home or via the sampled person’s current care facility. Model patient recruitment scripts are included as Attachments 9a and 9b. Project areas can modify these scripts to meet their specific needs. Unless the CDC model scripts are modified, additional OMB approval will not be sought for modifications made by individual project areas. The individual project area modifications will likely be minor.
All patient interviews (Attachment 2) will be conducted by trained CSBS staff in a private location either as part of a routine visit to a medical facility, or by an interview at home, in a hospital or clinic, or other mutually agreed upon location. Interviews may also be conducted over the telephone. The entire interview is expected to last for approximately 45 minutes.
The CSBS interview instrument (Attachment 2) will be provided by CDC in a Computer Assisted Personal Interview format so that data will be collected electronically. The interview will be administered face-to-face or through the telephone using electronic handheld devices or computers. The interview instrument was developed using Questionnaire Development System (QDS) software (NOVA Research Company, Bethesda, Maryland).
Participants will receive prevention materials at the end of the interview, referrals to local prevention and care services, and also prevention information from the CSBS staff, as requested.
In order to avoid data loss, and to ensure data security, at the end of each field visit the interviewers will be responsible for downloading and saving all data records into the local database. Once the downloading has occurred, all patient records should be deleted from the data collection computer’s hard drive before leaving for the next interview.
Medical record abstraction (Attachments 3a, 3b, 3c, and 3d) will be conducted by local project staff trained in the abstraction of clinical variables from medical charts. Standardized software on a laptop computer will be used for medical record abstraction. The information to be collected will be primarily related to diagnosis of opportunistic illnesses, provision of preventive therapies, prescription of antiretroviral medications, adverse events due to medications, and health services utilization.
Minimal data on all sampled patients from the HIV/AIDS Reporting System [HARS] (OMB Control No. 0920-0573: Adult and Pediatric Confidential HIV/AIDS Case Reports for National HIV/AIDS Surveillance) will be extracted using a computer program run by CSBS staff in each project area (Attachment 4). In rare cases in which a sampled patient cannot be located in HARS, information on patient demographics may be obtained from HIV care facility records. Minimal data on respondents and non-respondents will be compared to assess non-response bias. In addition, demographic data collected will be used for quality control purposes to ensure that patients are not sampled more than once.
The personally identifying information used to select patients will not be collected on the completed data collection forms; instead, each person will be assigned a unique ID.
The tablet and laptop computers used for data collection will be password protected and the data on them will be encrypted using standard, 128-bit encryption software. No personal identifiers will be collected or included. All data will be downloaded onto a secure computer at the health department and deleted from the field computers upon return to the office from the field.
Quality Control
For quality assurance purposes, a 5% subset of interviews will be observed by the project coordinator to determine accuracy and completeness. Additionally, interviewers will have periodic peer review of interviews to ensure the consistency in administration techniques across interviewers.
CDC will regularly train the interviewers and convene lessons learned meetings to understand the problems that can occur with the software and hardware that is used for conducting the interviews. Training topics will include how to use the CDC-provided software and hardware, conduct the interviews, archive the collected data, and transfer the data. CDC will also provide a manual with detailed instructions on interview conduct to participating state and local health departments.
Automated edit checks will be built into the computer software programs as a further quality control measure.
CDC is responsible for overseeing the development and distribution of the medical record abstraction software program to the participating state and local health departments. CDC will conduct abstractor training, and also provide a manual with detailed instructions for data abstraction to participating state and local health departments.
CDC will ensure regular training of abstractors and convene lessons learned meetings to understand the problems that can occur with the software and hardware that are used for conducting the abstraction. Automated edit checks will be built into the computer software programs as a further quality control measure.
Completed CSBS electronic abstraction records (Attachments 3a, 3b, 3c, and 3d) will be visually scanned to check for completeness. A 5% subset of medical records will be re-abstracted by a second, independent reviewer and compared to the original abstraction forms to determine completeness and discrepancies. The medical records selected for re-abstraction should be from a variety of facilities, abstractors, and time periods.
CDC conducts at least one site visit to each grantee per cycle. The purpose of the site visit is to monitor adherence to the project protocol, observe interviews and medical record abstractions, and obtain feedback on study procedures. Additional site visits specific to the CSBS demonstration project will be conducted as needed.
CSBS will use precisely the same methods for maximization of response rates and for dealing with nonresponse as the OMB-approved data collection—MMP (0920-0740, expires 5/31/2015). Because the CSBS interview takes approximately 45 minutes to administer, contains sensitive questions, and a significant portion of the population of HIV-infected adults in care are members of racial and ethnic minorities, patients will be offered remuneration for their participation to increase response rates. Participants will receive approximately $25 in cash for participation in the interview. If local regulations prohibit cash reimbursement, equivalent reimbursement may be offered in the form of personal gifts, gift certificates, or bus or subway tokens.
Research indicates that providing remuneration to respondents helps raise response rates for long, sensitive, in-person surveys (Kulka 1995). In addition, persons at risk for HIV infection have frequently been the focus of health-related data collections, in which remuneration is the norm (Thiede 2009; MacKellar 2005). Research has shown that financial incentives are effective at increasing response rates among female residents in minority zip codes (Whiteman 2003). A meta-analysis of 95 studies published between January 1999 and April 2005 describing methods of increasing minority enrollment and retention in research studies found that incentives enhanced retention among this group (Yancy 2006). Data from MMP’s 2007 cycle indicate that 65% of respondents reported a race or ethnicity other than non-Hispanic white. Providing remuneration to CSBS respondents is critical to achieve acceptable response rates.
Reimbursement is also provided to persons who participate in CDC’s HIV-related data collections among other populations, such as the National HIV Behavioral Surveillance System (NHBS) (OMB 0920-0770, exp. 3/31/2014) and the Transgender HIV Behavioral Survey (OMB No. 0920-0794, exp. 12/31/2010). Reimbursement was also used in the Supplement to HIV/AIDS Surveillance (SHAS) project (OMB 0920-0262, exp. 06/30/2004) (described in A.1.), for persons who agreed to participate in the interview. Participants were offered $25 as reimbursement for their time.
The same advisory boards that provide input into MMP will provide input on CSBS. A national provider advisory board, made up of providers of HIV care, provides input on MMP (and will provide input on CSBS) to CDC. A national community advisory board (CAB), made up of community members from each project area, serves as a link between MMP staff and patients who participate, and will also serve as a link between CSBS staff and participants. The national CAB shares information about the project and provides feedback to CDC about patient recruitment, data collection, and how the project is perceived by the community. Input from these two groups help to maximize facility and patient response and minimize patient non-response.
Like MMP, CSBS will use telephone interviewing as an optional mode for questionnaire administration in order to increase response rates. Use of mixed mode for survey administration has been found to result in improved response rates (de Leeuw 2005). In addition, conference calls between CDC and the project areas will be held on a monthly basis to review response rates and provide technical assistance to improve patient and facility response.
Assessing Non-Response Bias
The same procedures for assessing non-response bias that are currently used for MMP will be used for CSBS. Minimal data on all sampled patients from NHSS will be extracted using a computer program run by CSBS staff in each project area (Attachment 4). Minimal data on respondents and non-respondents will be compared to identify predictors of non-response. Those predictors with statistically significant effects will be used in the development of weight adjustment classes. Along with selection probabilities based on the sampling design, non-response data will factor into calculation of analytic weights so as to increase the generalizability of the information obtained to the universe of HIV-diagnosed adults.
These methods will be based on the assessment of non-response bias that has been completed for the 2009 MMP data collection cycle. In those analyses, the most significant predictors of patient response were facility size, race/ethnicity, years since diagnosis and age group. The ability to assess and adjust for nonresponse is a strength of probability surveys that may compensate for lower than desired response rates (Groves 2006).
CSBS recruitment will be monitored through on-going data reports generated weekly and monthly from the data submitted to CDC. The field staff and CDC will use the data in these reports to identify problems with recruitment. When a problem with response or recruitment arises during data collection, field staff will be instructed to consult with local stakeholders and facility staff to identify solutions to the problem.
The CSBS pilot is intended to answer key questions about the feasibility of case-surveillance-based sampling methods for use by the national MMP surveillance system. For example:
What criteria should be used to select individuals from the NHSS database?
What methods are most useful to locate and recruit persons sampled through NHSS?
What response rates may be expected when using case-surveillance-based sampling?
What barriers to case-surveillance-based sampling are identified and what solutions should be explored?
In addition, the CSBS pilot will test new questions designed for individuals not currently receiving HIV care, i.e. persons ineligible for standard MMP. The CSBS interview was developed using questions from the MMP interview. New questions added to the CSBS interview were reviewed internally and cognitively tested with fewer than 10 respondents.
CDC staff tested the skip patterns and responses both electronically and using paper versions of the data collection instruments.
The medical record abstraction application used for CSBS will be identical to that used for MMP. Mock medical records were developed to serve as training aids to the data abstractors. CDC staff also used the mock medical records to test the data abstraction instrument. OMB will be informed of any changes to data collection procedures or instruments as quickly as possible.
Consultants on Statistical Aspects
The following individuals consulted on statistical aspects only. They are not involved in collecting or analyzing the data.
ICF Macro:
Tonja M. Kyle, M.A.
Senior Manager
ICF Macro
11785 Beltsville Drive, Suite 300
Calverton, MD 20705
301.572.0820
301.572.0986 (f)
tkyle@icfi.com
Ronaldo Iachan, Ph.D
Senior Statistician
ICF Macro
11785 Beltsville Drive, Suite 300
Calverton, MD 20705
301.572.0820
301.572.0986 (f)
rIachan@icfi.com
Individuals Collecting and/or Analyzing Data
CDC is not directly engaged with human subjects during data collection. However, CDC Project Staff below will train health department staff in data collection methods, monitor the progress of recruitment by health department staff, and analyze the data.
CDC Project Staff
All CDC project staff can be reached at the following address and phone number:
Behavioral and Clinical Surveillance Branch
Division of HIV/AIDS Prevention
Centers for Disease Control and Prevention
1600 Clifton Rd, NE MS E-46
Atlanta, GA 30333
Phone: (404) 639-2090
Jacek Skarbinski, MD Team Leader Clinical Outcomes Team Dvo5@cdc.gov
|
Christine Mattson, PhD Epidemiologist Email: ggi8@cdc.gov
|
Linda Beer, PhD Epidemiologist Email: lbeer@cdc.gov
|
Sandra Stockwell, RN Nurse Consultant Email: sstockwell@cdc.gov
|
Janet Blair, PhD MPH Epidemiologist Email: jblair@cdc.gov
|
Stanley Wei, MD, MPH Medical Epidemiologist Email: bge3@cdc.gov |
Catherine Sanders, MA Public Health Advisor Email: hge3@cdc.gov
|
John Weiser, MD, MPH Medical Epidemiologist Email: eqn9@cdc.gov |
Ann Do, MD, MPH Medical Epidemiologist Email: ado@cdc.gov
|
Lydia Poromon, MPH Public Health Advisor Email: fks9@cdc.gov
|
Jennifer Fagan, MA Behavioral Scientist Email: jafagan@cdc.gov
|
Jeanne Bertolli, PhD, MPH Associate Chief for Science, Behavioral and Clinical Surveillance Branch Email: JBertolli@cdc.gov
|
Emma Frazier, PhD Epidemiologist Email: elf3@cdc.gov
|
James Heffelfinger, MD, MPH Chief, Behavioral and Clinical Surveillance Branch Email: JHeffelfinger@cdc.gov
|
Christopher Johnson, MS Statistician Email: cjohnson@cdc.gov |
McKaylee Robertson, MPH ORISE Fellow Email: img7@cdc.gov |
The following contracted staff will analyze CSBS data.
ICF International CDC CIMS Contract Project Staff
All CDC CIMS contracted staff can be reached at the following address and phone number:
Behavioral and Clinical Surveillance Branch
Division of HIV/AIDS Prevention
Centers for Disease Control and Prevention
1600 Clifton Rd, NE MS E-46
Atlanta, GA 30333
Phone: (404) 639-2090
Stella Chuke Data Manager Slc7@cdc.gov |
Shetul Shah Data Manager Gwq5@cdc.gov
|
Ping Huang Data Manager Hyv0@cdc.gov
|
Bertram Thomas Data Manager Bct7@cdc.gov
|
Glenn Nakamura Data Manager Gcn5@cdc.gov
|
Yan Zhang Data Manager Vtt3@cdc.gov
|
Roshni Patel Data Manager Jqe6@cdc.gov
|
|
CDC personnel responsible for receiving and approving CIMS contract deliverables:
Roseanne English
Associate Director for Data Management
Division of HIV/AIDS Prevention
REnglish-Bullard@cdc.gov
ICF International Data Coordinating Center Contract
All Data Coordinating Center contracted staff can be reached at the following address and phone number:
ICF International
11785 Beltsville Drive, Suite 300
Calverton, MD 20705
Phone: (800) 393-5936
Gia Badolato SAS Programmer
|
Mirna Moloney, MS Statistician mmoloney@icfi.com |
Baibai Chen, MA Senior Lead SAS Programmer
|
Stephanie Richelsen, MA Technical Assistance Coordinator
|
Christian Evans, MA, M Div Deputy Project Director
|
Walter Rives, MA Technical Assistance Coordinator
|
Deirdre Farrell, MPH Analytical Epidemiologiest
|
Luz Rodriguez Technical Assistance Coordinator
|
Lee Harding, MS Statistician
|
Pedro Saavedra, PhD Senior Statistical Advisor
|
Ronaldo Iachan, PhD Senior Statistical Team Lead
|
Joe Singh SAS Programmer
|
Kamya Khanna Jr. SAS Programmer
|
Wen Song, MS SAS Programmer
|
Tonja Kyle, MS Project Director
|
|
CDC personnel responsible for receiving and approving Data Coordinating Center contract deliverables:
Alicia Edwards
Health Scientist
Division of HIV/AIDS Prevention
Aje0@cdc.gov
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | ziy6 |
| File Modified | 0000-00-00 |
| File Created | 2021-02-03 |