Supporting Statement B
Ryan White HIV/AIDS Program Outcomes and Expanded Insurance Coverage Collections of Information Employing Statistical Methods
Statistical methods will not be used in this data collection.
1. Respondent Universe and Sampling Methods
The RWHAP funds over 1,300 sites. Thus, a sample of sites will be asked to participate in the project. The 25 RWHAP-funded sites involved in the site interview will be drawn from the 30 sites involved in HAB’s previous special study (also conducted by Abt) focusing on the effect of the changing healthcare landscape conducted by Abt Associates. The original 30 sites were identified and selected using the Ryan White HIV/AIDS Program grant recipients and provider site list provided by HAB to create a purposive sample with a range of provider settings that are geographically distributed across the U.S and address the diversity of Ryan White HIV/AIDS Program grant recipients and their clients. Also, involving the previous set of sites will allow the study team to draw upon the previous site interview data already collected for 2013-2014, thus reducing the level of burden on sites to gather historical data while simultaneously improving the level of data quality by reducing recall bias.
The 15 client cases per site for medical and administrative records abstraction will be selected from the client case load at the 25 study sites using a stratified sample based upon type of healthcare coverage selected.
Online survey invitations for site surveys will be sent to the universe of HAB’s 775 RWHAP-funded OAMC provider sites; based on past experience, we expect approximately 40% of sites (305 in all) to complete the survey.
On-site client focus groups will be conducted by the study team at up to six of the 25 study sites. We will ask each of these sites to gather a convenience sample of up to 10 clients with differing insurance profiles.
The following table describes statistical analyses performed to develop the sampling design and its level of generalizability for each of the data sources.
Quantitative/Qualitative Methods Employed Analytic Approaches to be Applied |
|
Data Sources |
Analyses |
Quantitative Methods: All quantitative data will be entered into SAS-compatible databases for analysis. Quantitative analyses will use a threshold of p<0.05 for determining statistical significance. |
|
RSR/ADR Client Data |
Sampling: We will obtain RSR/ADR client data for the universe of sites providing 10 or more outpatient ambulatory care (OAMC) visits in the most recently available year of data (estimated to include approximately 775 sites in all). Data will extend from 2012 through 2015 to allow examination of trends before and after January 2014. (We will additionally extract 2016 data if the study timeline permits.)
Measures: The RSR client data include measures of viral load, insurance type, and service utilization. The ADR data include measures of ADAP enrollment/disenrollment, level and type of prescription cost-sharing support, and ART prescriptions.
Analysis: We will conduct a difference-in-differences analysis (a.k.a. pre-post with comparison group) to assess differences in trends between sites in before and after January 2014. Subgroup analyses will examine differences in effects for client subpopulations (e.g. by insurance type). Models will be estimated using ordinary least squares (OLS) regression for continuous outcomes (e.g. viral load), Poisson or negative binomial regressions for count outcomes (e.g. number of OAMC visits), and logistic regression for binary outcomes (e.g. viral suppression). Standard errors will be adjusted to account for clustering of clients within sites.
Generalizability: Results will be generalizable to the universe of RWHAP sites providing OAMC services. |
Site Survey |
Sampling: We will invite the universe of sites providing 10 or more OAMC visits per year (as indicated in the latest available RSR data) to complete a web survey. Approximately 775 sites will receive the survey invitation, and we project at least 305 complete responses (or roughly a 40% response rate). Nonresponse weights will be constructed to account for differential response by provider type, size, and geographic location.
Measures: Sites will assess health outcomes by health care coverage type; cost sharing burden by coverage type; RWHAP services provided to insured and Medicaid-covered clients to fill gaps in care; services most difficult for newly covered clients to access; coverage limits: level of services offered, lack of providers, challenges to accessing a PCP; level of access and utilization by service type and group; core medical and support services that contribute most to retention and suppression and moving clients along the HIV Care Continuum; relative pharmaceutical coverage and associated cost-sharing for client; types of ART drugs available through insurance, e.g., newer single tablet regimens vs. combination therapies; and client switching between ADAP/LPAP and insurance pharmaceutical coverage.
Analysis: We will perform descriptive tabulations and cross-tabulations of survey responses. Statistically significant differences between groups will be assessed using t tests for continuous measures and chi square tests for categorical measures. All analyses will incorporate nonresponse weights.
Power: Assuming at least 305 completed surveys, we will be sufficiently powered to support a margin of error of approximately ±5 percentage points for binary measures.
Generalizability: Weighted results will be generalizable to the universe of RWHAP sites providing OAMC services. |
Medical Charts/Administrative Records Abstraction |
Sampling: We have purposively selected 25 sites with a range of provider settings that are geographically distributed across the U.S., a mix of Medicaid expansion and non-Medicaid expansion states, mirror the epidemic, and address the diversity of RWHAP Recipient programs and their clients. Within each of these 25 sites, we will perform medical and administrative records abstraction for a stratified random sample of 15 clients who were uninsured prior to the implementation of expanded health care coverage options, for a total of 375 clients in all. Stratification will be by current coverage (uninsured, Medicaid, or private insurance) with equal numbers of clients sampled in each coverage stratum to maximize power for comparing outcomes across groups. We will produce sampling weights accounting for differential probability of selection by coverage type.
Measures: Measures collected via our abstraction tool will include health outcomes data related to HIV and other overall health concerns, e.g., cardiovascular disease, hypertension, diabetes, and obesity; insurance/coverage type; service utilization; and pharmaceutical prescriptions.
Analysis: We will conduct a difference-in-differences analysis (a.k.a. pre-post with comparison group) to assess differences in trends for clients within sites before and after January 2014. Subgroup analyses will examine differences in effects for client subpopulations (e.g. by insurance type). Models will be estimated using ordinary least squares (OLS) regression for continuous outcomes (e.g. viral load), Poisson or negative binomial regressions for count outcomes (e.g. number of OAMC visits), and logistic regression for binary outcomes (e.g. viral suppression). Standard errors will be adjusted to account for clustering of clients within sites.
Power: Comparing two client groups of equal size, we will have sufficient power to detect a difference across groups of approximately 12.1 percentage points in the proportion of clients with high viral load.
Generalizability: Because abstractions will be performed within a purposive sample of sites, weighted results will be generalizable only to the 25 sites included in the analysis. |
Qualitative Methods: All qualitative data will be entered into NVivo to allow for standardized coding by topic and theme. A codebook will be developed and tested using a kappa of >.80 as an acceptable target for inter-coder reliability |
|
Site Interview. |
NVivo output will be used to:
The findings from the qualitative analyses will contextualize the quantitative results by reflecting on providers’ perceptions of facilitators and barriers affecting RWHAP clients’ access to services and service utilization, as well as health outcomes, specifically:
|
Focus Groups
|
NVivo output will be used to:
The findings from the qualitative analyses will contextualize the quantitative results by reflecting on clients’ perceptions of facilitators and barriers affecting their access to services and service utilization, as well as health outcomes, specifically:
|
2. Procedures for the Collection of Information
Medical Chart/Records Abstraction: On-Site
Site visit study staff will conduct the medical chart/records abstraction during 1.5 day site visits using a secure electronic web-based abstraction tool containing predefined data entry fields, as well as a free text notes section and will employ business specifications developed promote data quality. After the data has been entered, the study staff will immediately upload the data into Abt’s secure servers the using a secure FTP. The data will be in a format appropriate for import into SAS and the creation of an analytic file.
Site Interviews: Telephone and Face-to-Face
The Abt Team will interview healthcare providers and site administrators from selected RWHAP-funded program sites. Appropriate persons will be identified through recruitment calls by the project study staff with leadership at each site. The study team already has established relationships with each of these sites. Each interview will include one healthcare provider and one administrator per site. Interviews conducted via telephone will use a toll-free conference call number and will be joined remotely by an interviewer, note-taker, and interviewee(s). Interviews conducted face-to-face will be attended in-person by an interviewer, note-taker, and interviewee(s). For both telephone and face-to-face interviews, detailed notes will be taken during the course of the interviews, to be reviewed, coded in NVivo, and analyzed following the interview. Site interviews will utilize purposive sampling and are not intended to be statistically representative or generalizable to all RWHAP-funded program sites.
Site Focus Groups: Face-to-Face
The Abt Team will conduct focus groups with RWHAP clients at selected RWHAP-funded program sites. Appropriate persons will be recruited by RWHAP provider site staff. Each focus group will include up to 10 clients. The focus group process will include a facilitator, note-taker(s), and participants. Detailed notes will be taken during the course of the focus groups, to be reviewed, coded in NVivo, and analyzed following the focus groups. Site focus groups will use convenience sampling and are not intended to be statistically representative or generalizable to all RWHAP-funded program sites.
Site Surveys: Online
We will send survey invitations to 775 OAMC provider sites for the site surveys. HRSA Project Officers (PO) will email the Site Directors of all of their RWHAP grantees sites with the link to the survey encouraging them to complete it. The survey will be programmed using FluidSurveys. Each provider will be assigned a user name and password, allowing them to complete the survey all at once or over multiple sessions if they wish. The Abt Team will take steps to increase response rate, including a streamlined and straightforward site survey, engagement of POs and grantees, and a team of recruiters to encourage participation. Along with Abt Team efforts, FluidSurveys has a tool to track response rates and send reminders to complete the survey. Finally, results can be analyzed in real time as FluidSurveys has tools to aggregate data, compare responses, and create reports immediately. These surveys will allow for data that will be generalizable to all RWHAP-funded program sites
3. Methods to Maximize Response Rates and Deal with Nonresponse
The Abt Team will be conducting the interviews, surveys, and focus groups using updated contact information, and therefore expects limited, if any, non-response due to an inability to locate sample members. The interview guide, survey, and focus group guide were developed with consideration to length and comprehension level so it is appropriate for staff and clients to complete. HRSA anticipates achieving a 90 percent or better response rate for all site interviews, records abstraction and focus groups. HRSA anticipates a 40% response rate for the site surveys.
Site Interviews: Non-response will be limited by recruiting only engaged sites to participate. The majority of the sites participated in a previous RWHAP study and actively voiced interest in participating in this study as well.
Site Surveys: The survey will be programmed into FluidSurveys, which has multiple features that facilitate user response. It allows for free text, multiple response questions, and complex question grids that combine various approaches. We find that question grids often shorten the survey while reducing burden for respondents. FluidSurveys also permits skip logic to streamline survey response and question seeding (i.e., pre-populating surveys with user-specific information that can be overwritten by the respondent). We will assign each provider a user name and password, allowing then to complete the survey all at once or over multiple sessions. FluidSurveys also has a tool to track response rates and send reminders to complete the survey. Finally, project officers, grantees, and a team of recruiters will encourage participation to minimize non-response.
Focus Groups: Focus group participants will include individuals recruited by providers with whom they already have existing relationships. The providers will assist with recruiting engaged and reliable clients to limit non-response.
4. Tests of Procedures or Methods to be Undertaken
The Abt Team will pre-test the data collection tools; the medical records abstraction tool will be pre-tested with five cases at a single site, the site interview guide will be pre-tested with a single site, the focus group guide will be pre-tested internally, and the site survey will be pre-tested with up to three sites. The overarching goals of the pre-tests will be to refine the wording and flow, increase efficiency, and assist with burden estimates. Comments provided will be incorporated into revised versions of the data collection tools.
5. Estimates of Annualized Hour and Cost Burden
The total burden for the individual for data collection participation is estimated at 60 minutes for medical records sample selection guides (i.e., Site Administrators), 120 minutes for Site Interviews (i.e., Site Administrators and Healthcare Providers), 90 minutes for focus groups (i.e., clients), and 30 minutes for site surveys (i.e., Site Administrators). Time estimates are based on experience with similar instruments in other studies of comparable organizations.
The site surveys will be conducted at months 14 to 16. Medical Chart/Records Abstraction, site interviews, and focus groups will be conducted at months 16 to 18 of the project period.
Exhibit 1: Total Estimated Annualized Burden - Hours
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Total Responses |
Average Burden per Response (in hours) |
Total Burden Hours |
Site Survey |
305 |
1 |
305 |
0.5 |
152.5 |
Medical Chart/Record Abstraction |
25* |
1 |
25 |
2 |
50 |
Focus Group (recruit participants) |
25* |
1 |
25 |
1 |
25 |
Site Interview Guide |
50 |
1 |
50 |
2 |
100 |
Focus Groups Guide |
60 |
1 |
60 |
1.5 |
90 |
Total |
440* |
|
440* |
|
417.5 |
*The same respondents will complete the medical chart/record abstraction and recruit participants for the focus group.
6. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Marlene Matosky, MPH, RN
Contracting Officer Representative (COR)
Health Resources & Services Administration
HIV/AIDS Bureau
Division of Data and Policy
5600 Fishers Lane, Room 09N176B
Rockville, Maryland 20857
Phone: 301.443.0798
Email: mmatosky@hrsa.gov
Role: Oversees design of data collection plan, collection of data, and data analysis
Please note: the following individuals are either employed by the organization contracted to conduct project or are employed by an organization subcontracted to conduct project.
Liza Solomon, DrPH, MHS
Principal Associate, Abt Associates
4550 Montgomery Ave., Suite 800 North
Bethesda, MD 20814
Phone: 301.347.5785
Email: Liza_Solomon@abtassoc.com
Role: Senior Advisor Team Lead
Michael Costa, MPH
Principal Associate, Abt Associates
55 Wheeler St.
Cambridge, MA 02138
Phone: 617.349.2873
Email: Michael_Costa@abtassoc.com
Role: Project Director and Qualitative Analytic Team Lead
Caitlin Murphy, MPH
Project Manager
4550 Montgomery Ave., Suite 800 North
Bethesda, MD 20814
Senior Analyst, Abt Associates
Phone: 301.347.5356
Email: Caitlin_Murphy@abtassoc.com
Role: Project Manager and Data Collection and Management Lead
Lauren Olsho, PhD
Senior Associate, Abt Associates
55 Wheeler St.
Cambridge, MA 02138
Phone: 617.520.2326
Email: Lauren_Olsho@abtassoc.com
Role: Quantitative Analytic Team Lead
Chris Flygare, MA
Associate, Abt Associates
55 Wheeler St.
Cambridge, MA 02138
Phone: 617.349.2302
Email: Chris_Flygare@abtassoc.com
Role: Quantitative Data Manager
Alex Mijares, MPH
Senior Analyst, Abt Associates
2200 Century Parkway, Suite 950
Atlanta, GA 30345
Phone: 404.946.6378
Email: Alex_Mijares@abtassoc.com
Role: Qualitative Data Manager and Site Interviewer
Tana Brummer, BA
Analyst, Abt Associates
55 Wheeler St.
Cambridge, MA 02138
Phone: 617.520.2691
Email: tana_brummer@abtassoc.com
Role: Site Interviewer
Jennifer Carter, MPH
Senior Analyst, Abt Associates
4550 Montgomery Ave., Suite 800 North
Bethesda, MD 20814
Phone: (301) 347-5640
Email: Jennifer_Carter@abtassoc.com
Role: Site Interviewer
Diane Fraser, BA
Web Project Manager, Abt Associates
55 Wheeler St.
Cambridge, MA 02138
Phone: 617.520.2513
Email: Diane_Fraser@abtassoc.com
Role: Site Interviewer
Alyssa Kogan, BA
Associate Analyst, Abt Associates
55 Wheeler St.
Cambridge, MA 02138
Phone: 617.520.2309
Email: Alyssa_Kogan@abtassoc.com
Role: Site Interviewer
Joanna Lopez, BS
Associate Analyst, Abt Associates
4550 Montgomery Ave., Suite 800 North
Bethesda, MD 20814
Phone: 301.634.1880
Email: Joanna_Lopez@abtassoc.com
Role: Site Interviewer
Leigh Matias, MPH
Associate, Abt Associates
2200 Century Parkway, Suite 950
Atlanta, GA 30345
Phone: 404.946.6374
Email: Leigh_Mathias@abtassoc.com
Role: Site Interviewer
Mira Levinson, MPH
John Snow, Inc.
44 Farnsworth Street
Boston, MA 02210
Phone: 617.482.9485
Email: mlevinson@jsi.com
Role: JSI Team and Qualitative Lead
Karen Schneider, PhD
John Snow, Inc.
44 Farnsworth Street
Boston, MA 02210
Phone: 617.482.9485
Email: kschneider@jsi.com
Role: JSI Quantitative Researcher and Site Interviewer
Jeanne Day, MPH
Project Director, John Snow, Inc.
44 Farnsworth Street
Boston, MA 02210
Phone: 617.482.9485
Email: jday@jsi.com
Role: Medical Chart/Records Abstraction Lead and Site Interviewer
Nancy Reinhalter, RN, CCRC
Senior Research Coordinator, John Snow, Inc.
1616 Fort Myer Dr, Ste 1100
Arlington, VA 22209
Phone: 703.528.7474
Email: nancy_reinhalter@jsi.com
Role: Medical Chart/Records Abstraction Trainer and Site Interviewer
Molly Higgins-Biddle MPH
Consultant, John Snow, Inc.
44 Farnsworth Street
Boston, MA 02210
Phone: 617.482.9485
Email: mhiggins@jsi.com
Role: Medical Chart/Records Abstraction
Joseph Rego, BS
Project Associate, John Snow, Inc.
44 Farnsworth Street
Boston, MA 02210
Phone: 617.482.9485
Email: jrego@jsi.com
Role: Medical Chart/Records Abstraction
Katelyn Flaherty Doré, MPH
Project Manager, John Snow, Inc.
44 Farnsworth Street
Boston, MA 02210
Phone: 617.385.3920
Email: katelyn_flaherty_dore@jsi.com
Role: Medical Chart/Records Abstraction
Peggy O-Brien-Strain, PhD
President and Principal Associate, Mission Analytics Group, Inc.
Phone: 415.263.9885
Email: mstrain@mission-ag.com
Role: Mission Team Lead and Provides guidance on data analysis
Elizabeth Coombs, MPP
Research Associate, Mission Analytics Group, Inc.
Phone: 415.796.0159
Email: ecoombs@mission-ag.com
Role: Mission Project Director
Eric Verhulst, BA
Senior Analyst, Mission Analytics Group, Inc.
Phone: 415.796.0166
Email: everhulst@mission-ag.com
Role: Data Analyst and Programmer
Aloka Narayanan
Analyst, Mission Analytics Group, Inc.
415-814-1557
Role: Programmer
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Abt Single-Sided Body Template |
| Author | Administrator |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |