Ssb

Red Carpet Entry (RCE) Program Implementation Project

OMB No. 0920-New

February 18, 2022

Supporting Statement

Part B

CONTACT

Deborah Gelaude, M.A.
Technical Monitor

Centers for Disease Control and Prevention
Division of HIV Prevention, HRB
1600 Clifton Road, NE, Mailstop US8-5
Atlanta, GA 30329
Phone: 404-639-1905
Fax: 404-639-0897
E-mail: DGelaude@cdc.gov

TABLE OF CONTENTS

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods 1

2. Procedures for the Collection of Information 3

3. Methods to Maximize Response Rates and Deal with No Response 9

4. Tests of Procedures or Methods to Be Undertaken 10

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 10

EXHIBITS

Exhibit B1. Estimated Recruitment Targets 2

Exhibit B2. Summary of RCE Implementation Staff Recruitment Targets 2

Exhibit B3. Study Population by Aim 3

Exhibit B4. Data Types and Sources 3

Exhibit B5. Data Types Used to Address Study Aims 4

Exhibit B6. RTI Staff ……………………………………………………………………………10

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

This information collection request is for individual in-depth interview and online survey data from ten staff implementing the Red Carpet Entry (RCE) program in two clinics over six months to evaluate the implementation of RCE in those clinics. Online labor/non-labor cost questionnaires will also be completed by a staff member from each clinic. Data will also include information from 126 consenting patient records (appointment, demographic, and date of diagnosis data). The study population will be the clinic study staff and leaders and clinic clients who were referred to either of the two clinics and who are (1) newly diagnosed with HIV or (2) re-entering HIV care. The overall purpose of the study is to evaluate the implementation of RCE using a toolkit created by the Contractor and other SMEs and retain RCE’s program objective of linking patients to HIV care within 72 hours. RCE is a structural level intervention developed by Whitman Walker Health and the District of Columbia Department of Health and implemented in Washington, DC since 2009.

1. Respondent Universe and Sampling Methods

Target Population and City Selection

We will conduct the project at two clinical sites, Rutgers New Jersey Medical School Infectious Disease Practice and Florida Department of Health in Hillsborough County, to implement the RCE intervention and participate in the project. These sites were selected because (1) they each serve as a U.S. Office of Management and Budget–designated metropolitan jurisdiction prioritized in the first phase of the U.S. Department of Health and Human Services’ Ending the HIV Epidemic in the U.S. (EHE) initiative, (2) they have existing partnerships with organizations working in HIV prevention and treatment that are willing to act as local stakeholders and provide CTR Counselors, and (3) they have the skills and experience to successfully perform the requirements of this study based on past performance with RTI.

An estimated ten staff members and 126 RCE Clients will be included in this study:

RCE Clients (n=126). RCE clients are persons newly diagnosed with HIV or persons with HIV who have been out of HIV care for at least 12 months. Based on the expected number of newly diagnosed and return-to-care clients per month at each RCE site, up to 126 clients are expected to be enrolled over six months.

Implementation Staff (n=8). Four key staff members at each site—the Clinic Champion, the RCE Concierge, the internal clinic CTR Counselor, and the external referral clinic CTR Counselor. Staff fulfilling four key RCE implementation roles at each site will participate in the surveys and interviews, for a total of eight implementation staff across the sites.

Clinic Leadership (n=2). Clinic leadership includes staff at the clinical sites with the decision-making power to institute systems-level changes necessary to implement RCE, such as policy changes, who are not implementation staff. One clinic leader will be identified at each clinic site to complete the stakeholder interviews.

Sample Estimates

Based on estimates provided by the clinics, the potential RCE client respondent population is 331 for the RCE implementation project (see Table B1). It is estimated that about 38% of the total respondent sample will participate in the RCE program. Participants will be sampled using a convenience strategy. Newly diagnosed or returned to care persons will be told about the program and will be invited to participate. This process will continue until we have reached 63 participants at each clinic.

Exhibit B1. Estimated Recruitment Targets

Clinic	Number of newly diagnosed patients seen/year	Number of returning to care patients seen/year	Total potential N/6 months	RCE 6-month recruitment and implementation N
Florida Department of Health, Hillsborough County Specialty Care Clinic	314	71	193	63
Rutgers New Jersey Medical School Infectious Disease Practice	131	145	138	63
Totals	445	216	331	126

Exhibit B2. Projected Implementation Staff and Clinic Leadership sample size for each clinic

Site	Implementation Staff	Clinic Leadership	RCE Clients
Rutgers	4	1	63
Hillsborough	4	1	63
TOTAL	8	2	126

Data will be collected from consenting RCE Clients, implementation staff, and clinic leaders. The study population of each aim is described in Exhibit B3.

Exhibit B3. Study Population by Aim

Aim	Study Population
Aim 1: Describe the RCE Implementation Process	Implementation staff, clinic leadership
Aim 2: Evaluate Implementation Contexts and Outcomes	Implementation staff, clinic leadership
Aim 3: Evaluate Costs	Implementation staff
Aim 4: Evaluate Health Service–Related Outcomes	RCE Clients
Aim 5: Evaluate the Implementation Toolkit	Implementation staff

2. Procedures for the Collection of Information

Each aim of the study will be evaluated using the procedures as shown in Exhibits B3, B4, and B5. For Aims 1, 2, and 5, one survey and one interview will be completed per timepoint by each implementation staff member to assess all three aims.

Exhibit B4. Data Types and Sources

Data Type	Description	Data Source
Process evaluation data	De-identified data collected by the RCE Concierge to manage the RCE intervention, including referrals received, referral sources, and RCE components received by the clients.	The RCE Concierge will routinely collect these data during the implementation period using a secure, password-protected Microsoft Access database stored on their clinic’s secure server or project computer. The data manager will transfer de-identified data to RTI monthly.
Implementation staff interviews	De-identified transcripts of interviews with key implementation staff.	RTI will audio-record interviews with key implementation staff and create de-identified transcriptions.
Implementation staff surveys	Online survey data collected from key implementation staff.	These data will be collected online using Qualtrics.
Stakeholder interviews	De-identified transcripts of interviews with key system-level stakeholders.	RTI will audio-record interviews with stakeholders and create de-identified transcriptions.
Cost data	Labor and non-labor cost data.	Cost data will be collected using Excel workbooks and information obtained from invoices and budgets submitted to RTI.
Appointment, demographic, and date of diagnosis data	Appointment, demographic, and date of diagnosis data extracted from the clinic’s EHR for consenting RCE Clients.	The data manager will extract appointment attendance data, demographic data, and date of diagnosis data from the clinic’s EHR and enter the data into a secure, password-protected Microsoft Access database stored on the clinic’s server or project computer. The data manager will transfer de-identified data to RTI every 2 months.
Baseline linkage to and retention in care data	Baseline linkage to and retention in care rates for the year prior to RCE implementation.	These data will come from the clinic’s EHR.

Exhibit B5. Data Types Used to Address Study Aims

Study Aim	Data Type(s)
Aim 1: Describe the RCE Implementation Process	Implementation staff interviews, implementation staff surveys, staff and leadership interviews
Aim 2: Evaluate Implementation Contexts and Outcomes	Process evaluation data, implementation staff interviews, implementation staff surveys, staff and leadership interviews
Aim 3: Evaluate Costs	Cost questionnaire
Aim 4: Evaluate Health Service–Related Outcomes	Appointment, demographic data, and date of diagnosis data
Aim 5: Evaluate the Implementation Toolkit	Implementation staff interviews, implementation staff surveys

Process Evaluation Data. The RCE Concierge at each clinic will document and manage activities related to the RCE Program in a secure, password-protected Microsoft Access database stored on their clinic’s servers or clinic-owned computer. The RCE Concierge will record information related to RCE referrals, scheduled appointments, receipt of RCE core components at appointments, and clients’ consent to share their appointment data with RTI. The RCE Concierge will record identifying information such as the client’s clinic identifier (e.g., medical record number), the client’s name, and appointment dates for program management purposes. However, no identifying information will be sent to RTI. De-identified client-level data will be sent to RTI every month during the implementation period and a Microsoft Access-generated report aggregating these metrics will be sent to RTI biweekly. All referred clients will receive a unique study ID to track their data, with the link between their study ID and clinic identifier maintained at the clinics in their Microsoft Access database. In addition, RTI will provide written performance feedback in the form of bimonthly reports cards. The report cards will be informed by information collected through RTI’s programmatic check-ins with the sites, and ad hoc TA. The RCE sites will use the report cards to adjust their implementation procedures as needed. Within 10 days of production, RTI will provide CDC with each report card.

Implementation Staff Surveys. Key clinic staff implementing RCE (i.e., the Clinic Champion, RCE Concierge, internal CTR Counselor, and external CTR Counselor) at each clinic will complete a 15-minute online survey once during the pre-implementation period (see Attachment 3a) and each month during the implementation period (see Attachments 3b and 3c). The survey will have closed-ended questions assessing implementation strategies used, acceptability, appropriateness, feasibility, implementation climate, implementation readiness, and organizational readiness for implementing change. Survey data will be collected using Qualtrics and downloaded as Excel files to RTI servers for analysis.

Implementation Staff Interviews. Individual interviews will be conducted with key clinic staff implementing RCE who also completed the staff surveys. Using the semi-structured interview guide (see preparation phase interview guide in Attachment 3d and implementation phase guide in Attachments 3e and 3f), an RTI staff member will lead the interview, and an additional staff member will take notes. The preparation phase interview will take approximately 1 hour to complete; all other interviews will take approximately 30 minutes to complete. All interviews will be digitally recorded on RTI devices, transcribed, and entered in NVivo for analysis.

Clinic Leadership Interviews. Individual interviews will be conducted with one member of clinic leadership at each site. Using the semi-structured interview guide (see Attachment 3g), an RTI staff member will lead the 30-minute interview, and an additional staff member will take notes. All interviews will be digitally recorded on RTI devices, transcribed, and entered in NVivo for analysis.

Cost Data. Information on the amount of time staff spend on program activities during pre-implementation and implementation phases will be collected using online labor questionnaires. The Clinic Champion, RCE Concierge, and internal CTR Counselor from each clinic will be asked to complete the labor questionnaire four times: (1) after the final month of pre-implementation, (2) after the second month of implementation, (3) after the fourth month of implementation, and (4) after the sixth month of implementation. Staff will be asked to report time spent on each program activity during a typical week of the month for which the data are reported.

The RCE Concierge will be responsible for collecting all the required data for the labor questionnaire from other staff members and will submit the data to RTI in an Excel workbook (see Attachment 3h). RTI will obtain information on staff salaries and fringe benefits from the budgets and invoices submitted by the clinic or by contacting the RCE Concierge. The RCE Concierge will also be responsible for completing the non-labor cost questionnaire (see Attachment 3i) in an Excel workbook monthly during the pre-implementation and implementation phases. The non-labor cost questionnaire asks staff to record all indirect or overhead costs and materials, travel, services, and equipment expenditures incurred during the previous month. Cost data for any materials purchased by RTI for use in the clinics will be tracked at RTI.

RTI will obtain additional data needed for the cost analysis, such as the salaries and fringe benefits of program staff and indirect/overhead costs from other sources of data, specifically clinic budgets and invoices submitted to RTI.

Appointment, Demographic, and Date of Diagnosis Data. Appointment, demographic, and date of diagnosis data will be collected routinely through the clinics’ existing EHR. If the clinic’s appointment scheduling system is separate from their EHR, data will also be obtained from the scheduling system. The data manager at each site will extract data from their clinic systems and enter data for consenting RCE Clients into a Microsoft Access database throughout the implementation period. The clinics will transfer de-identified appointment data to RTI every two months.

2a. Eligibility

Implementation staff/Clinic Leadership:

• Employed in an RCE program role

• No exclusion criteria

Eligible RCE Clients:

• 18 years of age or older OR an emancipated minor

• Diagnosed with HIV

• English- and/or Spanish-reading/speaking

• Attend a visit at one of the two implementing clinics for HIV care

• Consent to share their appointment, demographic, and date of diagnosis data

• Meet one of the following:

  • Be newly diagnosed with HIV

  • Be new to care

  • Be out of care (last attended appointment at the clinic was more than 12 months

ago)

Exclusion Criteria

Potential respondents will be excluded from the study if they are unable or unwilling to provide consent for any reason, unable to speak or read English/Spanish, are unable to attend a participating clinic, or are not: over 18 or an emancipated minor, HIV-positive.

Justification for Exclusion of Population Segments

This research focuses on persons with HIV who are newly diagnosed or returning to HIV care after 12 months. Persons without HIV are excluded from the study as this would not benefit them.

Emancipated minors (e.g., individuals under 18 who are able to give their consent to participant in a study) are eligible for inclusion in this study. RTI is including emancipated minors because (1) there is minimal risk to participants and (2) RCE is designed to become the standard of care at implementing clinics, so all individuals who are able to consent to health care should be eligible for RCE. Each clinical site will follow the professional standards of clinical care for patients, including any other types of participants who might be classified as vulnerable, such as pregnant women.

2b. Procedures for Consent, Recruitment, and Enrollment

RCE Clients. There are two ways a patient can participate in the RCE program: 1) newly diagnosed HIV-positive or 2) identified as out of HIV care. If a patient does not meet one of the above criteria, or is not interested in participating, they will be marked as ineligible for the RCE implementation evaluation but will still receive the clinic’s standard of care services.

Newly diagnosed and out-of-care PWH will be recruited into the RCE Program through internal (clinic-based) and external (CBO) partners, and via social media advertisements (see Attachment 6a). Clients will be provided with a palm card that includes a description of RCE (see Attachment 6b). The palm cards will also give new clients who are uncomfortable requesting HIV-related services a discrete password (“Red Carpet Entry”) that they can use to request service at the clinic. All staff members who have initial contact with the public, such as receptionists, triage staff, and unit clerks, will be aware of the password and how to connect the client with the RCE Concierge.

In addition, clients will also be able to self-refer with instructions provided on social media advertisements. The ads will include the RCE Concierge’s contact information and the password to request RCE service (“Red Carpet Entry”). Clinic sites will also be provided with these social media marketing materials and will have the option to place advertisements on these and other social media platforms to reach potentially eligible PWH.

Once the RCE Concierge receives a referral, they will contact the client as soon as possible, confirm eligibility, and schedule an appointment for the client to be seen at the clinic within 72 hours of referral.

The RCE Concierge will greet RCE Clients upon their arrival at the clinic and initiate the RCE visit. All RCE Clients will be asked to consent to sharing their appointment, demographic, and date of diagnosis data (to evaluate health service–related outcomes). The consenting process will be done in a way that is cognizant of the heightened emotions and information overload clients are likely experiencing during their initial RCE visit. To maintain participant anonymity, RTI will not collect signed informed consent forms, but they will be retained by the clinics only. Each clinic will submit a version of the written consent form (Attachments 4a and 4b) and Health Insurance Portability and Accountability Act of 1996 (HIPAA) Authorization Form (Attachments 4c and 4d) to their clinic’s Institutional Review Board (IRB) prior to the implementation of the study. The consent form and HIPAA Authorization Form apply to the transmission of the clients’ clinic attendance, demographic, and date of diagnosis data.

Although clients may be approached in a public area within the clinic, the RCE Concierge will bring the client into a private space for the informed consent process. The RCE Concierge or other RCE staff will review the written consent form with the client. To ensure that consent is voluntary, the RCE staff will allow time for the participant to ask questions, the client will be informed that their clinical care is not contingent upon their involvement in the study and that they are able to refuse. The consent document is in plain language (Attachments 4a and 4b). The RCE staff member will ask the client if they consent to participate, and if the client agrees, the client will sign the consent form and will receive a copy for their records. The participant will be asked to sign the HIPAA Authorization Form at the time of consent to authorize the clinic to release their de-identified medical records to RTI, specifying the data elements that clinic staff will abstract from the EHR and send to RTI. RTI will provide the clinics with an authorization form, or they can choose to use their own. The clinical sites will retain the written consent and HIPAA authorization forms so that RTI does not receive any participant names. The RCE Concierge will record the date and time at which the client consented and record it in the clinic’s Microsoft Access database. The key file linking the client’s clinic identifier (e.g., EHR ID) to their study ID will be stored in the password-protected Microsoft Access database. All data will be collected from the patient’s electronic health record (EHR) or appointment scheduling records; no data will be collected directly from the clients. The client’s decision to participate in the study will not impact their participation in RCE.

The RCE Concierge will record all referrals in a Microsoft Access database to track and manage program implementation. They will record the date and time of the referral, the referral source, and the patient’s information. To evaluate program success, they will also document whether and when an appointment was scheduled and the RCE components that the client received at their visit.

If the RCE Concierge is unable to establish contact with a referred client or a client does not attend their RCE visit, the RCE Concierge will attempt outreach through a process called RCE Outreach and Reengagement.

Implementation Staff. Four staff members responsible for key RCE implementation roles (Clinic Champion, RCE Concierge, internal CTR Counselor, and external CTR Counselor) at each site will be recruited to participate in the staff interviews, staff surveys, and cost surveys. At the start of each online staff survey, implementation staff will be shown the survey consent form (see Attachment 4e). Staff will indicate their consent to participate in each of the online surveys by selecting “I agree to participate” and then taking and submitting the survey. At the start of each interview, RTI staff will obtain verbal consent from the participant at the beginning of the recorded in-depth interview using the script shown in Attachment 4f. RTI will not be collecting personal or sensitive data from implementation staff. Staff will be provided with a palm card that includes a description of RCE (see Attachment 6c).

Clinic Leadership. At the start of each of the interviews, RTI staff will obtain verbal consent from the participant at the beginning of the recorded in-depth interview using the script shown in Attachment 4g. RTI will not be collecting personal or sensitive data from clinic leadership.

We are requesting a 24-month approval. The preparation phase will begin with RTI staff engaging and training implementation staff three months prior to the start of implementation. The implementation phase will begin when RCE Concierges begin taking referrals. Data collection will continue to assess retention in care outcomes.

3. Methods to Maximize Response Rates and Deal with No Response

The investigators involved in this project have extensive experience recruiting and retaining clinic patients for projects involving the collection of behavioral and clinical data.

Achieving sufficient enrollment in the study is critical to the success of the RCE implementation project and we propose to offer a token of appreciation to all enrolled RCE Clients. Participants will be informed of the token of appreciation during the consenting process. Token of appreciation will be provided to each study participant at enrollment upon consent. The token of appreciation will be a $25 gift card. (Please see Statement A, Section A9 for a detailed description of the rationale and how the RCE study will administer tokens of appreciation.)

The RCE program is being implemented at each clinic, and for those enrolled, fully integrated as part of patients’ clinic appointment scheduling. Therefore, if enrolled patients are returning for their clinical appointments, they should also be retained in the study. The study will rely on each clinic’s standard of care for making appointment reminders.

Recruitment and retention will be monitored through ongoing data report cards and weekly calls. RTI, clinic, and CDC staff will use these data to identify issues with recruitment or retention. When a problem with recruitment or retention arises during data collection, research staff will be instructed to consult with clinic staff to identify solutions.

If the RCE Concierge is unable to establish contact with a referred client or a client does not attend their RCE visit, the RCE Concierge will attempt outreach through a process called RCE Outreach and Reengagement. The RCE Concierge will make two attempts to engage the client. If attempts are unsuccessful, the RCE Concierge will inform the CTR Counselor. The CTR Counselor will then make two or more attempts to engage clients. If contact is established during any of the four+ attempts, the RCE Concierge or the CTR counselor will reschedule the client’s appointment and address any barriers that led to the missed appointment. If attempts to reach the client by the RCE Concierge and the CTR Counselor fail, the client will be referred to the clinic’s standard of care outreach.

For Aim 2, clinic staff will be assured that their responses to the online survey and qualitative interviews will be kept private. Staff implementing RCE are briefed on the survey and interview activities and will participate as part of their regular duties. They will also provide informed consent to participate in the study. If implementation staff choose to not participate in data collection, the study staff will approach other implementation staff members in similar positions. Backup staff will be trained at each clinic and will also be involved in RCE implementation.

4. Tests of Procedures or Methods to Be Undertaken

The data collection elements were developed by RTI, with consultation from CDC and an Implementation Science expert (Brian Powell) and have been reviewed by the RCE CAB. Our team includes RCE program experts, research methods experts, including screening and instrument development, and experts in implementation research design and methods.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

CDC Consultant on Statistical Aspects

Craig Borkowf, PhD

Biostatistician

Division of HIV Prevention

U.S. Centers for Disease Control and Prevention

uzz3@cdc.gov

(404) 639-5235

Implementation Science Expert

Byron Powell, PhD

Washington University

One Brookings Dr.

Campus Box 1196

Saint Louis, MO 63130

Phone: 314-935-2817

bjpowell@wustl.edu

Contractor

CDC awarded a contract to RTI in 2020 to conduct the Red Carpet Entry Program Implementation Project. The RTI staff is involved with all aspects of designing and implementing the study.

Exhibit B6. RTI Staff

Olivia Burrus, MPH	Project Director: Responsible for overall scientific integrity for the project across all aims and fiscal management; primary contact for CDC
Brittany Zulkiewicz, MPH	Associate Project Director: Responsible for project management; assisting Principal Investigators with cooperative agreement management, site subcontracts, and consultant agreements; supervising other study research associates
Megan Lewis, PhD	Senior Advisor: Responsible for advising project team on design of study protocol, data collection, and interpretation
Shawn Karns, BA	Data Manager: Responsible for managing data for all sites and for analysis of Aim 4 data
Stephen Tueller	Lead Statistician: Responsible for leading quantitative Aim 1 data analysis
Bryan Garner, PhD	Senior Advisor: Responsible for advising project team on design of study protocol, data collection, and interpretation
Olga Khavjou, MA	Cost Data Lead: Responsible for cost data collection and analysis for Aim 2
Alexa Ortiz, MSN	Research Analyst: Responsible for outreach protocol and assisting the Principal Investigators and Associate Project Director with site management
Haley Hedrick, BA	Research Associate: Responsible for assisting Ms. Burrus with site administration and project organization

CDC Project Staff

The CDC staff members who are involved with the various aspects of designing and implementing the study are listed below. CDC staff will not be in contact with study participants. CDC will receive only study data with no information in identifiable form. The data collected will be analyzed by CDC staff. All CDC project staff can be reached at the following address and phone number:

HIV Research Branch

Division of HIV Prevention

Centers for Disease Control and Prevention

1600 Clifton Rd, NE MS US8-5

Atlanta, GA 30329

Phone: (404) 639-1900

Deborah Gelaude, MA, Technical Monitor

HIV Research Branch

Division of HIV Prevention

Carla Galindo, MPH, Contracting Officer’s Representative

HIV Research Branch

Division of HIV Prevention

Cari Courtenay-Quirk, PhD, Consultant

HIV Research Branch

Division of HIV Prevention