Supporting Statement B_11-1-17

Supporting Statement B

Assessing Client Factors Associated with Detectable HIV Viral Loads; and

Models of Care and the Ryan White HIV/AIDS Program

OMB Control No. 0906-XXXX-New

2. Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

A sample of 75 sites will be selected from the “universe” of the Ryan White HIV/AIDS Program (RWHAP) funded Outpatient Ambulatory Health Services (OAHS) provider sites that complete the web survey administered as part of the Ryan White HIV/AIDS Program Outcomes and Expanded Insurance Coverage study (Ryan White Outcomes) (OMB#: 0906-0030). This web survey will be sent to the Health Resources and Services Administration (HRSA), HIV/AIDS Bureau’s (HAB) 775 RWHAP-funded OAHS provider sites. It is anticipated that the survey response rate will be at least 40%, resulting in a minimum of 305 responses. No site will be selected to participate in more than one of the RWHAP studies. The HIV Viral Suppression study will visit 25 of the 75 sites, and the Models of Care study will visit the remaining 50 sites.

Eighteen to twenty client cases per site for medical and administrative records abstraction will be selected from the client case load at the 75 study sites using a stratified sample based upon study focus (see tables below). In addition, as part of the HIV Viral Suppression study, 20 clients at the site visit clinics will be randomly selected for a survey based on their viral suppression status, and six of the 20 will be asked to participate in an interview.

Clients participating in the focus groups for the Models of Care study will be selected by the site staff. These site staff will be instructed to invite clients currently receiving care at the clinic in the last 18 months.

Provider staff participating in the face-to-face interviews will be identified by site administrators based on instruction from the HRSA contractor team. Providers should be those who are most familiar with and/or most often provide care to RWHAP clients.

The following table describes statistical analyses performed to develop the sampling design and its level of generalizability for each of the data sources in the HIV Viral Suppression study.

Assessing Client Factors Associated with Detectable HIV Viral Loads
Quantitative/Qualitative Methods Employed Analytic Approaches to be Applied
Data Sources	Analyses
Quantitative Methods: All quantitative data will be entered into SAS-compatible databases for analysis. Quantitative analyses will use a threshold of p<0.05 for determining statistical significance.
Ryan White HIV/AIDS Services Report (RSR)/ AIDS Drug Assistance Program (ADAP) Data Report (ADR) (RSR/ADR) Client Data	Sampling: We will obtain RSR/ADR client data for the universe of sites providing 10 or more outpatient ambulatory care (OAMC) visits in the most recently available year of data (estimated to include approximately 775 sites in all). Based on the 2014 RSR data, we expect approximately 337,832 client records in all. Measures: The RSR client data include measures of client demographics, health outcomes related to HIV and other primary care conditions, insurance type, and service utilization. The ADR data include measures of ADAP enrollment/disenrollment, level and type of prescription cost-sharing support, and antiretroviral treatment (ART) prescriptions. Analysis: We will conduct a correlational analysis to assess site- and individual-level characteristics associated with viral suppression. Models will be estimated using logistic regression with an indicator of viral suppression status as the dependent variable. Standard errors will be adjusted to account for clustering of clients within sites. Power: Assuming 775 sites, and a design effect of 1.113 due to clustering of clients within sites, consistent with the 2014 RSR data, our sample of 337,832 clients will yield power for a bivariate analysis to detect a difference in viral suppression of approximately 0.5 percentage points across two equally-sized client groups. Generalizability: Results will be generalizable to the universe of RWHAP sites providing OAMC services.
Site Survey	Sampling: We will invite the universe of sites providing 10 or more OAMC visits per year (as indicated in the latest available RSR data) to complete a web survey. Approximately 775 sites will receive the survey invitation, and we project at least 305 complete responses (or roughly a 40% response rate). Nonresponse weights will be constructed based on the full RSR dataset to account for differential response by provider type, size, and geographic location. Measures: We will assess sites’ model of care delivery for RWHAP patients based on services offered at the site; dominant type of providers used at the site to care for HIV patients; type and roles of staff employed by the site; and level of collaboration and integration among HIV, primary or other specialty providers at or off the site. Sites will also report health outcomes by health care coverage type; cost sharing burden by coverage type; RWHAP services provided to insured and Medicaid-covered clients to fill gaps in care; services most difficult for newly covered clients to access; coverage limits: level of services offered, lack of providers, challenges to accessing a PCP; level of access and utilization by service type and group; core medical and support services that contribute most to retention and suppression and moving clients along the HIV Care Continuum; relative pharmaceutical coverage and associated cost-sharing for clients with different types of health care coverage; types of ART drugs available through insurance, e.g., newer single tablet regimens vs. combination therapies; and client switching between ADAP/Local Pharmaceutical Assistance Program (LPAP) and insurance pharmaceutical coverage. Analysis: We will perform descriptive tabulations and cross-tabulations of survey responses. Sites will be grouped into tertiles by rate of viral load suppression as observed in the RSR data. Statistically significant pairwise differences between site-level characteristics across the three viral suppression rate tertiles will be assessed using t tests for continuous measures and chi square tests for categorical measures. All analyses will incorporate nonresponse weights. Power: With a sample of 305, we would have power to detect a pairwise difference of approximately 19.6 percentage points across viral suppression tertiles for a binary characteristic with a prevalence of 50% in one of the tertiles. Power will be improved relative to this benchmark for characteristics that are more or less prevalent. Generalizability: Weighted results will be generalizable to the universe of RWHAP sites providing OAMC services.
Medical Charts/Administrative Records Abstraction	Sampling: We will use stratified random sampling to select 25 RHWAP sites from among the universe of sites providing 10 or more OAMC visits per year as identified in the most recent available RSR data, anticipated to be approximately 775 sites in all. Stratification factors will be at minimum geographic region and urbanicity; we will sample proportionately within each stratum to ensure adequate representation. Within each of these 25 sites, we will perform medical and administrative records abstraction for a stratified random sample of up to 18 clients for a total of 450 clients in all. Client-level stratification will be by viral suppression status (suppressed or not suppressed) with equal numbers of clients sampled in each stratum to maximize power for comparing outcomes across the two groups. We will produce sampling weights accounting for the complex sampling design at site and client levels. Measures: Measures collected via our abstraction tool will include health outcomes data related to HIV and other overall health concerns, e.g., cardiovascular disease, hypertension, diabetes, and obesity; insurance/coverage type; service utilization; and pharmaceutical prescriptions. Analysis: We will conduct a correlational analysis to assess site- and individual-level characteristics associated with viral suppression. Models will be estimated using logistic regression with an indicator of viral suppression status as the dependent variable. Standard errors will be adjusted to account for clustering of clients within sites. All analyses will incorporate sampling weights. Power: Assuming 25 sites, and a design effect of 1.113 due to clustering of clients within sites, consistent with the 2014 RSR data, our sample of 450 clients will allow us to detect differences between suppressed and non-suppressed clients of approximately 14.7 percentage points for a binary characteristic with a prevalence of 50% in one of the groups. Power will be improved relative to this benchmark for characteristics that are more or less prevalent. Generalizability: Weighted results will be generalizable to the universe of RWHAP sites providing OAMC services.
Client Survey	Sampling: Within each of the 25 selected sites (see above), site staff will assist us in purposively selecting up to 20 clients to complete client surveys, preferably evenly divided between suppressed and non-suppressed clients. Measures: This survey will elicit information about perceived stigma, medical mistrust, past trauma, HIV knowledge and socio-demographic information. Analysis: We will conduct a correlational analysis to assess site- and individual-level characteristics associated with viral suppression. Models will be estimated using logistic regression with an indicator of viral suppression status as the dependent variable. Standard errors will be adjusted to account for clustering of clients within sites. Power: Assuming 25 sites, a response rate of 85%, and a design effect of 1.113 due to clustering of clients within sites, consistent with the 2014 RSR data, our projected sample of 425 clients will allow us to detect differences between suppressed and non-suppressed clients of approximately 15.1 percentage points for a binary characteristic with a prevalence of 50% in one of the groups. Power will be improved relative to this benchmark for characteristics that are more or less prevalent. Generalizability: Because clients will be identified purposively, results of this analysis should not be considered generalizable beyond the sampled population.
Qualitative Methods: All qualitative data will be entered into NVivo to allow for standardized coding by topic and theme. A codebook will be developed and tested using a kappa of >.80 as an acceptable target for inter-coder reliability
Site Interview	NVivo output will be used to: Identify types of common themes Frequency of similar or divergent perceptions reported by RWHAP service providers. The findings from the qualitative analyses will contextualize the quantitative results by reflecting on providers’ perceptions of facilitators and barriers affecting RWHAP clients’ access to services, service utilization, and ability to achieve and maintain viral suppression, specifically: Whether and how client rates of core medical and support services affect RWHAP clients’ rates of viral suppression. Which barriers to engagement and retention in care most affect RWHAP clients’ ability to achieve or maintain viral suppression. Which psycho-social factors and social determinants of health (e.g., SES, self-efficacy, stigma, trauma, medical mistrust, food insecurity, transportation) most affect the RWHAP clients’ ability to achieve or maintain viral suppression. What clinical factors most affect RWHAP clients’ ability to achieve or maintain viral suppression (e.g. co-morbidities, time since HIV diagnosis, ARV regimen [type, initiation, exposure length]). What factors influence clients’ ability to overcome challenges and achieve viral suppression. Experience with successful interventions employed by providers to: Link clients to the services Combat social determinates of health in treating clients with a detectable viral load Increase adherence to medications
Client Interview	NVivo output will be used to: Identify types of common themes Frequency of similar or divergent perceptions reported by RWHAP clients. The findings from the qualitative analyses will contextualize the quantitative results by reflecting on clients’ perceptions of facilitators and barriers affecting their access to services, service utilization, and ability to achieve and maintain viral suppression, specifically: How use of core medical and support services affect the RWHAP clients’ ability to achieve and maintain viral suppression. Which barriers to engagement and retention in care most affect the RWHAP clients’ ability to achieve or maintain viral suppression. Which psycho-social factors and social determinants of health (e.g., SES, self-efficacy, stigma, trauma, medical mistrust, food insecurity, transportation) most affect RWHAP clients’ ability to achieve or maintain viral suppression. What clinical factors most affect RWHAP clients’ ability to achieve or maintain viral suppression (e.g. co-morbidities, time since HIV diagnosis, ARV regimen [type, initiation, exposure length]). What factors influence clients’ ability to overcome challenges and achieve viral suppression. What are the barriers or challenges experienced by medical providers and support service providers helping clients to achieve and maintain viral suppression. What are some successful interventions employed by providers to: Link clients to the services Combat social determinates of health in treating clients with a detectable viral load Increase ARV use among clients who initially refuse ARV Increase adherence to medications

The following table describes statistical analyses performed to develop the sampling design and its level of generalizability for each of the data sources in the Models of Care study.

RW Models of Care and the RWHAP
Quantitative/Qualitative Methods Employed Analytic Approaches to be Applied
Data Sources	Analyses
Quantitative Methods: All quantitative data will be entered into SAS-compatible databases for analysis. Quantitative analyses will use a threshold of p<0.05 for determining statistical significance.
Site Survey	Sampling: We will invite the universe of sites providing 10 or more OAMC visits per year (as indicated in the latest available RSR data) to complete a web survey. Approximately 775 sites will receive the survey invitation, and we project at least 305 complete responses (or roughly a 40% response rate). Nonresponse weights will be constructed to account for differential response by provider type, size, and geographic location. Nonresponse weights will be constructed based on the full RSR dataset to account for differential response by provider type, size, and geographic location. Measures: We will assess sites’ model of care delivery for RWHAP patients based on services offered at the site; dominant type of providers used at the site to care for HIV patients; type and roles of staff employed by the site; and level of collaboration and integration among HIV, primary or other specialty providers at or off the site. Sites will also report health outcomes by health care coverage type; cost sharing burden by coverage type; RWHAP services provided to insured and Medicaid-covered clients to fill gaps in care; services most difficult for newly covered clients to access; coverage limits: level of services offered, lack of providers, challenges to accessing a PCP; level of access and utilization by service type and group; core medical and support services that contribute most to retention and suppression and moving clients along the HIV Care Continuum; relative pharmaceutical coverage and associated cost-sharing for clients with different types of health care coverage; types of ART drugs available through insurance, e.g., newer single tablet regimens vs. combination therapies; and client switching between ADAP/LPAP and insurance pharmaceutical coverage. Analysis: We will perform descriptive tabulations and cross-tabulations of survey responses. Sites will be grouped by model of care (primary, specialist, or integrated). Statistically significant pairwise differences between outcomes associated with the three models of care will be assessed using t tests for continuous measures and chi square tests for categorical measures. All analyses will incorporate nonresponse weights. Power: Assuming a sample of 305 sites evenly distributed across the three models of care, we will have power to detect a pairwise difference of approximately 19.6 percentage points across models for a binary outcome with a prevalence of 50% in one of the models. Power will be improved relative to this benchmark for outcomes that are more or less prevalent. Generalizability: Weighted results will be generalizable to the universe of RWHAP sites providing OAMC services.
RSR/ADR Client Data	Sampling: We will obtain RSR/ADR client data for the universe of sites providing 10 or more outpatient ambulatory care (OAMC) visits in the most recently available year of data (estimated to include approximately 775 sites in all). However, our analytic sample will include only client data for the approximately 305 sites completing the site survey, allowing us to classify them by model of care. Based on the 2015 RSR data, we expect approximately 132,953 client records in all for the sites that respond to the survey. Site-level nonresponse weights will be constructed based on the full RSR dataset to account for differential survey response by provider type, size, and geographic location. Measures: The RSR client data include measures of client demographics, health outcomes related to HIV and other primary care conditions, insurance type, and service utilization. The ADR data include measures of ADAP enrollment/disenrollment, level and type of prescription cost-sharing support, and ART prescriptions. Analysis: We will conduct a correlational analysis to assess differences in HIV and other health outcomes across different models of care delivery. Subgroup analyses will examine differences in effects for client subpopulations (e.g. by insurance type). Models will be estimated using ordinary least squares (OLS) regression for continuous outcomes (e.g. viral load), Poisson or negative binomial regressions for count outcomes (e.g. number of OAMC visits), and logistic regression for binary outcomes (e.g. viral suppression). Standard errors will be adjusted to account for clustering of clients within sites. Power: Assuming an equal distribution of models of care across 305 sites, and a design effect of 1.113 due to clustering of clients within sites, consistent with the 2014 RSR data, our sample of 132,953 clients will yield power to detect a difference of approximately 0.8 percentage points for pairwise comparisons across models of care in the percent of clients with high viral load. Generalizability: Weighted results will be generalizable to the universe of RWHAP sites providing OAMC services.
Medical Charts/Administrative Records Abstraction	Sampling: We will use stratified random sampling to select 50 RHWAP sites from among the sites responding to the site survey (projected to be approximately 305 sites in all). Stratification will be by model of care (primary, specialty, or integrated) with equal numbers of clients sampled in each group to maximize power for comparing outcomes across the three models, paired with proportional sampling by geographic region and urbanicity to ensure adequate representation. Within each of the 50 selected sites, we will perform medical and administrative records abstraction for a random sample of up to 18 clients served in the prior 12 months for a total of 900 clients in all. We will produce sampling weights accounting for unequal probabilities of selection by site. Measures: Measures collected via our abstraction tool will include health outcomes data related to HIV and other overall health concerns, e.g., cardiovascular disease, hypertension, diabetes, and obesity; insurance/coverage type; service utilization; and pharmaceutical prescriptions. Analysis: We will conduct a correlational analysis to assess differences in health outcomes across the three different models of care. Subgroup analyses will examine differences in effects for client subpopulations (e.g. by insurance type). Models will be estimated using ordinary least squares (OLS) regression for continuous outcomes (e.g. viral load), Poisson or negative binomial regressions for count outcomes (e.g. number of OAMC visits), and logistic regression for binary outcomes (e.g. viral suppression). Standard errors will be adjusted to account for clustering of clients within sites. Power: Assuming an equal distribution of models of care across the 50 selected sites, and a design effect of 1.113 due to clustering of clients within sites, consistent with the 2014 RSR data, our sample of 900 clients will yield power to detect a difference of approximately 9.6 percentage points for pairwise comparisons across models of care in the percent of clients with high viral load. Generalizability: Weighted results will be generalizable to the universe of RWHAP sites providing OAMC services.
Qualitative Methods: All qualitative data will be entered into NVivo to allow for standardized coding by topic and theme. A codebook will be developed and tested using a kappa of >.80 as an acceptable target for inter-coder reliability.
Provider Interview	NVivo output will be used to: Identify types of common themes Frequency of similar or divergent perceptions reported by RWHAP service providers. The findings from the qualitative analyses will contextualize the quantitative results by reflecting on providers’ assumptions of what model of care is employed by their clinic and its impact on client outcomes in HIV and other chronic conditions; as well perceptions of facilitators and barriers affecting care coordination across HIV and primary care services, specifically: What models of care are used by RWHAP clinical care providers Whether and how staff roles and composition vary within each model of care Which models of care have better HIV clinical outcomes and other health outcomes Whether the effect of the models of care vary for different populations (e.g. among those with co-morbidities or by disease acuity, insurance, or subpopulations that currently experience poorer outcomes) What barriers or challenges (internal or external to the site) are inherent in each model of care, particularly with respect to care coordination How are comorbidities and other types of care effectively managed and/or integrated into the models of care Whether and how clients choose a specific model of care for their HIV and non-HIV medical and support services Whether and how stigma and provider cultural competency impact clients decision to access care via a specific model of care
Client Interview	NVivo output will be used to: Identify types of common themes Frequency of similar or divergent perceptions reported by RWHAP clients. The findings from the qualitative analyses will contextualize the quantitative results by reflecting on clients’ access to different models of care and its impact on achieving positive clinical outcomes, specifically: Whether the effect of the models of care vary for different populations (e.g. among those with co-morbidities or by disease acuity, insurance, or subpopulations that currently experience poorer outcomes) What barriers or challenges (internal or external to the site) are inherent in each model of care, particularly with respect to care coordination What are the barriers or challenges experienced by clients in accessing services in the different models of care How are comorbidities effectively managed in the different models of care Whether and how clients choose a specific model of care for their HIV and non-HIV medical and support services Whether and how stigma and provider cultural competency impact clients decision to access care via a specific model of care

2. Procedures for the Collection of Information

Provider Interviews: Face-to-face, on site

For both studies, the HRSA contractor will interview appropriate site staff from selected RWHAP-funded program sites. Appropriate persons will be identified through recruitment calls by the project study staff with leadership at each site. Each interview will include up to five participants per site. Interviews will be conducted face-to-face during the site visit by an interviewer and note-taker. Detailed notes will be taken during the course of the interviews, and will be reviewed, coded in NVivo, and analyzed following the interview. Audio recordings will be made only to serve as backup to the notes in Word. Provider interviews will utilize purposive sampling and are not intended to be statistically representative or generalizable to all RWHAP-funded program sites.

Client Survey and Semi-Structured Interviews (HIV Viral Suppression study sites only): Face-to-face, on site

For the HIV Viral Suppression study, the HRSA contractor will survey and interview clients from selected RWHAP-funded program sites (this data collection is not conducted for Models of Care). Appropriate persons will be recruited by RWHAP provider site staff. Up to 20 participants per site will be invited to complete a survey. Among those 20 individuals that complete the survey, six (6) clients will be randomly selected and asked to complete a semi-structured interview to gather more in-depth qualitative data. Surveys and interviews will be conducted in-person during the site visit by an interviewer. Detailed written notes will be taken during the course of the semi-structured individual interviews, and will be reviewed, coded in NVivo, and analyzed following the interview. Audio recordings will be made only to serve as backup to the notes in Word. Client surveys and interviews will utilize purposive sampling and are not intended to be statistically representative or generalizable to all RWHAP-funded program sites.

Medical Chart/Records Abstraction: On site

The HRSA contractor will conduct the medical chart/records abstraction during site visits using a secure electronic web-based abstraction tool that has already been developed by the contractor and reviewed as part of the Ryan White Outcomes study (OMB#: 0906-0030). The tool contains predefined data entry fields as well as a free text notes section, and will employ business specifications developed to promote data quality. After the data has been entered, the study staff will immediately upload the data into the contractor’s secure servers the using a secure file transfer protocol (FTP). The data will be in a format appropriate for import into SAS and the creation of an analytic file.

Site Focus Groups (Models of Care study sites only): Face-to-face, on site

For the Models of Care study, the HRSA contractor will conduct focus groups with RWHAP clients at 30 of the 50 RWHAP-funded program sites. Appropriate persons will be recruited by RWHAP provider site staff. Each focus group will include up to eight clients. The focus group process will include a facilitator, note-taker(s), and participants. Detailed notes will be taken during the course of the focus groups, and will be reviewed, coded in NVivo, and analyzed following the focus groups. Audio recordings will be made only to serve as backup to the notes in Word. Site focus groups will use convenience sampling and are not intended to be statistically representative or generalizable to all RWHAP-funded program sites.

3. Methods to Maximize Response Rates and Deal with Nonresponse

For the HIV Viral Suppression Study, the HRSA contractor will conduct the survey and interviews with clients attending routine appointments. For the Models of Care study, the HRSA contractor will work closely with clinic staff to identify interested and appropriate individuals.

The survey, interview and focus group guides were developed with consideration to length and comprehension level so it is appropriate for staff and clients to complete. HRSA anticipates achieving a 90 percent or better response rate for all provider and client interviews, client surveys, and focus groups.

Provider Interviews: Non-response will be limited by recruiting only engaged sites to participate.

Focus Groups/Client Interviews: Focus group and client interview/survey participants will include individuals recruited by providers with whom they already have existing relationships. The providers will assist with recruiting engaged and reliable clients to limit non-response.

4. Tests of Procedures or Methods to be Undertaken

The HRSA contractor will pre-test each of the data collection instruments for each study with fewer than 10 respondents across two sites. At one site, we will pre-test the provider and client interview guides and client survey for the HIV Viral Suppressions study; and at the second site, we will pre-test the provider interview guide and client focus group guide for the Models of Care study. The medical record abstraction tool will be pre-tested at both sites and will not be used for more than four (4) respondents at one site and five (5) respondents at the second site, for a total of 9 respondents across both sites. The overarching goals of the pre-tests will be to refine the wording and flow, increase efficiency, and assist with burden estimates. Comments provided will be incorporated into revised versions of the data collection tools.

6. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Pamela Klein, MSPH, PhD

Contracting Officer Representative (COR)

Health Resources & Services Administration

HIV/AIDS Bureau

Division of Data and Policy

5600 Fishers Lane

Rockville, Maryland 20857

Phone: 301.443.5545

Email: pklein@hrsa.gov

Role: Oversees design of data collection plan, collection of data, and data analysis

Robert Mills, PhD

Contracting Officer Representative (COR)

Health Resources & Services Administration

HIV/AIDS Bureau

Division of Data and Policy

5600 Fishers Lane

Rockville, Maryland 20857

Phone: 301.443.3899

Email: rmills@hrsa.gov

Role: Oversees design of data collection plan, collection of data, and data analysis

Jennifer Carter, MPH

Associate, Abt Associates

4550 Montgomery Ave., Suite 800 North

Bethesda, MD 20814

Phone: 301.347.5640

Email: Jennifer_Carter@abtassoc.com

Role: Project Director, Models of Care

Diane Fraser

Senior Communications Specialist, Abt Associates

55 Wheeler St.

Cambridge, MA 02138

Phone: 617.520.2513

Email: Diane_Fraser@abtassoc.com

Role: Project Manager, HIV Viral Suppression

Brandy Wyant, MPH

Senior Analyst, Abt Associates

55 Wheeler St.

Cambridge, MA 02138

Phone: 617.520.2834

Email: Brandy_Wyant@abtassoc.com

Role: Project Manager, Models of Care

Michael Costa, MPH

Principal Associate, Abt Associates

55 Wheeler St.

Cambridge, MA 02138

Phone: 617.349.2873

Email: Michael_Costa@abtassoc.com

Role: Senior Advisor, Models of Care; Project Director, HIV Viral Suppression

Chris Flygare, MA

Associate, Abt Associates

55 Wheeler St.

Cambridge, MA 02138

Phone: 617.349.2302

Email: Chris_Flygare@abtassoc.com

Role: Quantitative Data Manager for both studies

17